Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Get certified in Microsoft Fabric—for free! For a limited time, get a free DP-600 exam voucher to use by the end of 2024. Register now

Reply
frosty-fabric
Frequent Visitor

fabric data factory pipeline access on-premise data source

if i am not mistaken, the pipeline in fabric data factory cannot access on-premise data source still, is it on the road map, when we can expect this will not be an obstable?  data flow gen2 does support on-prem data source right now, only thing is from my testing, the performance of it is quite bad compared with if I use azure data factory. Hopefylly Fabric data factory pipeline will be a lot better later when it can connect to on-prem data source. Also, what about CI/CD ? I have ADF pipeline which copies data from on prem data source to data lake in fabric, next step is load it into fabric data warehouse, how do I connect these two step, since step 1 is ADF, it cannot see step 2 which is within Fabric. 

1 ACCEPTED SOLUTION
NandanHegde
Super User
Super User

Hey,

Couple of aspects :

1) Unlike PaaS version of ADF, Fabric Datapipeline and Dataflow gen 2 both use the Capacity as the support engine so depending on the capacity the performance might change.

 

The below link provides the roadmap :

https://learn.microsoft.com/en-us/fabric/release-plan/data-factory

 

and the plan seems to be in Q1 2024 for on prem support

 

2) Unfortunately the Data pipelines JSON code is read only as of today hence CICD is currently not possible but based on the road map w.r.t Git integrations, it would be available eventually

 

3) w.r.t 3rd ask, unfortuantly those 2 jobs would be in silos and you cannot directly integrate them because as of today Fabric doesnt have any REST API to trigger the pipelines externally and it doesnt support event/blob trigger as well.

 

 




----------------------------------------------------------------------------------------------
Nandan Hegde (MSFT Data MVP)
LinkedIn Profile : www.linkedin.com/in/nandan-hegde-4a195a66
GitHUB Profile : https://github.com/NandanHegde15
Twitter Profile : @nandan_hegde15
MSFT MVP Profile : https://mvp.microsoft.com/en-US/MVP/profile/8977819f-95fb-ed11-8f6d-000d3a560942
Topmate : https://topmate.io/nandan_hegde
Blog :https://datasharkx.wordpress.com

View solution in original post

4 REPLIES 4
v-cboorla-msft
Community Support
Community Support

Hi @frosty-fabric 

 

Glad that your query got resolved.

Please continue using Fabric Community for any help regarding your queries.

v-cboorla-msft
Community Support
Community Support

Hi @frosty-fabric 

 

Thanks for uisng Microsoft Fabric Community.

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others.
Otherwise, will respond back with the more details and we will try to help.


Thanks

Thanks for checking, I accepted a solution 

NandanHegde
Super User
Super User

Hey,

Couple of aspects :

1) Unlike PaaS version of ADF, Fabric Datapipeline and Dataflow gen 2 both use the Capacity as the support engine so depending on the capacity the performance might change.

 

The below link provides the roadmap :

https://learn.microsoft.com/en-us/fabric/release-plan/data-factory

 

and the plan seems to be in Q1 2024 for on prem support

 

2) Unfortunately the Data pipelines JSON code is read only as of today hence CICD is currently not possible but based on the road map w.r.t Git integrations, it would be available eventually

 

3) w.r.t 3rd ask, unfortuantly those 2 jobs would be in silos and you cannot directly integrate them because as of today Fabric doesnt have any REST API to trigger the pipelines externally and it doesnt support event/blob trigger as well.

 

 




----------------------------------------------------------------------------------------------
Nandan Hegde (MSFT Data MVP)
LinkedIn Profile : www.linkedin.com/in/nandan-hegde-4a195a66
GitHUB Profile : https://github.com/NandanHegde15
Twitter Profile : @nandan_hegde15
MSFT MVP Profile : https://mvp.microsoft.com/en-US/MVP/profile/8977819f-95fb-ed11-8f6d-000d3a560942
Topmate : https://topmate.io/nandan_hegde
Blog :https://datasharkx.wordpress.com

Helpful resources

Announcements
November Carousel

Fabric Community Update - November 2024

Find out what's new and trending in the Fabric Community.

Live Sessions with Fabric DB

Be one of the first to start using Fabric Databases

Starting December 3, join live sessions with database experts and the Fabric product team to learn just how easy it is to get started.

November Update

Fabric Monthly Update - November 2024

Check out the November 2024 Fabric update to learn about new features.

Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Early Bird pricing ends December 9th.