Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
tsunloan
Helper I
Helper I

On-prem data gateways are NOT supported by the Azure Function activity

On-premisis data gateway support for data pipelines has been announced as "shipped" however it on-prem gateways are still not supported by Azure Function activities.

 

The promise of delivery in Q1 2024 was not made with limits or caveats and it's perfectly sensible to assume that support would extend to a Azure services such as function apps rather than being strictly limited to the copy activity.

 

When will support be provided for connecting to Azure Functions in private networks via the Azure function pipeline activity?

 

Relevant links:
https://learn.microsoft.com/en-us/fabric/release-plan/data-factory#opdg

https://blog.fabric.microsoft.com/en-au/blog/integrating-on-premises-data-into-microsoft-fabric-usin...

 

14 REPLIES 14
tsunloan
Helper I
Helper I

In addition, on-prem asupport is not available in the "Web" pipeline activity.
API request are another perfectly valid and common way of retrieving data.

tsunloan_5-1713427578069.png

 

The declaration that on-prem data is supported in fabric pipelines should be accompanied by a strong disclaimer that on-prem support is strictly limited to the "Copy data" activity. 

R1k91
Solution Supplier
Solution Supplier

I didn't test it... 

maybe you can workaround it using dataflows? I mean calling the APIs from dataflows (that supports gateways) and orchestrating the dataflows call inside Pipelines?


--
Riccardo Perico
BI & Power BI Engineer @ Lucient Italia

Blog | GitHub

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

That's such a heavy handed approach for something that the pipeline activity should provide out-of-the-box.


It seems that nothing in Fabric actually works on it's own without elaborate workarounds or hacks.

Perhaps some of the huge marketing budget should be re-directed to engineering so they can build a working product before shilling it to corporate execs.

R1k91
Solution Supplier
Solution Supplier

I see you.

That sensation of driving in a road construction site..

Other approach I saw (especially for already working ADF projects) is to continue using ADF and self-hosted IR to load data into OneLake

How to ingest data into Fabric using the Azure Data Factory Copy activity - Microsoft Fabric | Micro...

again... it's a workaround


--
Riccardo Perico
BI & Power BI Engineer @ Lucient Italia

Blog | GitHub

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
tsunloan
Helper I
Helper I

My comment was very specific. On-prem gateways are not supported by the Azure Function activity in Fabric pipelines. Azure functions are a valid way of retrieving and transforming data.

 

tsunloan_0-1713426810736.png

tsunloan_1-1713426910411.png

tsunloan_3-1713427180725.png

 

tsunloan_2-1713426967091.png

 

From the Fabric roadmap:

tsunloan_4-1713427245002.png

The roadmap promised support for on-prem data in Q1 2024 and it does not say anywhere that on-prem support will only be available in the "Copy data" activity.

Anonymous
Not applicable

Hi @tsunloan ,

At this time, we are reaching out to the internal team to get some help on this .
We will update you once we hear back from them.

Anonymous
Not applicable

Hi @tsunloan ,

We have an update from internal team - 

" On-prem is only supported and limited towards Copy Activity but not in any other activities. "

Incase if your looking for a specific feature or functionality, I request you to please raise a feature request.

Appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

 

Hope this helps. Please let me know if you have any further queries.

 

That's bitterly disappointing.

I didn't think I needed to raise a feature request as the roadmap stated that support for on-prem data would be delivered in Q1 2024 and it didn't state anywhere that support was limited to the Copy activity.

 

I would have raised a feature request last year if they were honest about what they planned on delivering.

It's extremely dissapointing. We waited a year for OPDG support and they released it with very limited functionality....

And the functionality that was released doesn't even work.

Anonymous
Not applicable

Hi @tsunloan ,

I apologize for the issue, can you please help me understand what functionality is not working?
If you can share it with us, we will try to guide you.

Anonymous
Not applicable

Hi @tsunloan ,

We haven’t heard from you on the last response and was just checking back to see if you got a chance to look into my last response.
Can you please share the issue you are facing,  we will try to help .

Thanks

Anonymous
Not applicable

Hi @tsunloan ,

We haven’t heard from you on the last response and was just checking back to see if you got a chance to look into my last response.
Can you please share the issue you are facing,  we will try to help .

Thanks

Anonymous
Not applicable

Hi @tsunloan ,

Thanks for using Fabric Community.
As I can see we can now use on-prem data source in datapipeline - How to access on-premises data sources in Data Factory - Microsoft Fabric | Microsoft Learn

We can load the data from Onprem to cloud using Copy Activity.

Can you please help me understand with your specific requirment towards on-prem? What are you trying to achieve?
I will try to provide some help on your issue?

Helpful resources

Announcements
Sept Fabric Carousel

Fabric Monthly Update - September 2024

Check out the September 2024 Fabric update to learn about new features.

Expanding the Data Factory Forums

New forum boards available in Data Factory

Ask questions in Apache Airflow Job and Mirroring.

September Hackathon Carousel

Microsoft Fabric & AI Learning Hackathon

Learn from experts, get hands-on experience, and win awesome prizes.

Sept NL Carousel

Fabric Community Update - September 2024

Find out what's new and trending in the Fabric Community.