Forum Discussion
Connecting to a Rest API source - pipeline vs dataflow gen2
Hi,
I need to connect to a Dynamics 365 BC source by a Rest API.
I'm trying to implement and test a pipeline and a dataflow gen2.
I've noticed that a dataflow gen2 is more performant than a pipeline, but also significantly more expensive in CU: why?
Thinking to take advantage of dataflow gen2, does it exist any tricks to reduce the relative CU consumption?
Thanks
Hi pmscorca,
In general Dataflow Gen 2 is just more expensive.
You can see the pricing calculations here: https://learn.microsoft.com/en-us/fabric/data-factory/pricing-dataflows-gen2
Compared to the Pipeline CU costs: https://learn.microsoft.com/en-us/fabric/data-factory/pricing-pipelines
If you want the most cost effective approach, that would probably be to use a Python notebook to pull the data out of the REST API.
3 Replies
Hi pmscorca,
In general Dataflow Gen 2 is just more expensive.
You can see the pricing calculations here: https://learn.microsoft.com/en-us/fabric/data-factory/pricing-dataflows-gen2
Compared to the Pipeline CU costs: https://learn.microsoft.com/en-us/fabric/data-factory/pricing-pipelines
If you want the most cost effective approach, that would probably be to use a Python notebook to pull the data out of the REST API.
pmscorca Following up to confirm if the earlier responses addressed your query. If not, please share your questions and we’ll assist further
Thanks,
Lakshmi
