Power BI Dataflows or Azure Data Factory?

Question

Every day, you need to load 10GB of data both from on-prem instances of SAP ECC, BW and HANA to Azure DL Store Gen2. This is only the first step of a job that will continue to transform that data using Azure Databricks, Data Lake Analytics and Data Factory.What would you use for that load, Power BI (Premium) Dataflows or Azure Data Factory?How do you think to orchestrate the steps considering the use of Power BI Dataflows together with Azure Data Factory? Maybe using Azure Automation?&nbsp;It is a real use case.Thanks!

anonymous · Answer

Anonymous&nbsp;wrote:It would be perfect to have power query source in adf too.&nbsp;It would appear that this is coming:&nbsp;https://azure.microsoft.com/en-us/blog/analytics-in-azure-remains-unmatched-with-new-innovations/&nbsp;"A new capability called Wrangling Data Flows, available in preview, gives users the ability to explore and wrangle data at scale. Wrangling Data Flows empowers users to visually discover and explore their data without writing a single line of code."

tommartens · Answer

Hey,
&nbsp;
this is just my first thought.
It seems, that this process is not owned by a business user. From y first understanding I would use ADF to load your data into ADLS Gen2.
&nbsp;
Please be aware that currently Power BI expects a filesystem called powerbi inside the storage account:&nbsp;https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2
This might become useful in the future as you progress in your architecture.
&nbsp;
Regards,Tom

anonymous · Answer

Hi&nbsp;TomMartens,&nbsp;It means Dataflows are thought only for business users? Instead, I think that IT departments could use PBI workspaces only for certain mash-up processes that store the data in Azure Datalake. Extremely more rapid to develop than ADF. Any official thoughts about it? ...&nbsp;&nbsp;It would be perfect to have power query source in adf too. This would close the circle!&nbsp;Thanks!&nbsp;&nbsp;&nbsp;&nbsp;

michaelsh · Answer

Anonymous&nbsp;How did you eventually solve your challenge?I am facing a similar issue: adf vs. dataflows?Any conclusions?

jcarville · Answer

Interested myself in some comparisions between the two.

Forum Discussion

Related content

Data Factory - Dataflow gen2 Pricing in Fabric

Power BI dataflows

ETL Data Factory

dataflow gen1

Fabric Dataflow Gen 2