Starting December 3, join live sessions with database experts and the Microsoft product team to learn just how easy it is to get started
Learn moreGet certified in Microsoft Fabric—for free! For a limited time, get a free DP-600 exam voucher to use by the end of 2024. Register now
if i am not mistaken, the pipeline in fabric data factory cannot access on-premise data source still, is it on the road map, when we can expect this will not be an obstable? data flow gen2 does support on-prem data source right now, only thing is from my testing, the performance of it is quite bad compared with if I use azure data factory. Hopefylly Fabric data factory pipeline will be a lot better later when it can connect to on-prem data source. Also, what about CI/CD ? I have ADF pipeline which copies data from on prem data source to data lake in fabric, next step is load it into fabric data warehouse, how do I connect these two step, since step 1 is ADF, it cannot see step 2 which is within Fabric.
Solved! Go to Solution.
Hey,
Couple of aspects :
1) Unlike PaaS version of ADF, Fabric Datapipeline and Dataflow gen 2 both use the Capacity as the support engine so depending on the capacity the performance might change.
The below link provides the roadmap :
https://learn.microsoft.com/en-us/fabric/release-plan/data-factory
and the plan seems to be in Q1 2024 for on prem support
2) Unfortunately the Data pipelines JSON code is read only as of today hence CICD is currently not possible but based on the road map w.r.t Git integrations, it would be available eventually
3) w.r.t 3rd ask, unfortuantly those 2 jobs would be in silos and you cannot directly integrate them because as of today Fabric doesnt have any REST API to trigger the pipelines externally and it doesnt support event/blob trigger as well.
Glad that your query got resolved.
Please continue using Fabric Community for any help regarding your queries.
Thanks for uisng Microsoft Fabric Community.
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others.
Otherwise, will respond back with the more details and we will try to help.
Thanks
Thanks for checking, I accepted a solution
Hey,
Couple of aspects :
1) Unlike PaaS version of ADF, Fabric Datapipeline and Dataflow gen 2 both use the Capacity as the support engine so depending on the capacity the performance might change.
The below link provides the roadmap :
https://learn.microsoft.com/en-us/fabric/release-plan/data-factory
and the plan seems to be in Q1 2024 for on prem support
2) Unfortunately the Data pipelines JSON code is read only as of today hence CICD is currently not possible but based on the road map w.r.t Git integrations, it would be available eventually
3) w.r.t 3rd ask, unfortuantly those 2 jobs would be in silos and you cannot directly integrate them because as of today Fabric doesnt have any REST API to trigger the pipelines externally and it doesnt support event/blob trigger as well.
Starting December 3, join live sessions with database experts and the Fabric product team to learn just how easy it is to get started.
Check out the November 2024 Fabric update to learn about new features.
User | Count |
---|---|
6 | |
5 | |
3 | |
2 | |
2 |
User | Count |
---|---|
18 | |
11 | |
8 | |
7 | |
6 |