topic Re: dataflow in Pipelines

dataflow

Anonymous — Wed, 15 May 2024 19:07:31 GMT

Hello,

In azure ADF, previously I created a dataflow which reads a .csv sitting in the raw container of a storage account and then does a bit of transformations and then saves the data as delta parquet files into the curated container.

Now I would like to do something similar but in fabric.

What is the way to do this in fabric? I ask because I see in the lakehouse an option to store Files. So I tried it out. In Files I uploaded the .csv

perhaps in fabric dataflow I can read the .csv file and do transformation or maybe in Files folder I convert the .csv file to table?...

Any guidance please?

Thank you

Re: dataflow

Anonymous — Thu, 16 May 2024 03:38:43 GMT

Hello again Anonymous ,

Thanks for using Fabric Community.
Fabric doesn't directly support dataflows like Azure Data Factory (ADF) for data transformations.
In Fabric we have Data Flow Gen 2 inorder to do complex transformations from UI.

Docs to refer - Create your first Microsoft Fabric dataflow - Microsoft Fabric | Microsoft Learn

You can also also use Notebooks with pyspark to do these tranformations.

If you are looking to load csv file to table (without any transformations), you can simply upload the file and can use the option - Load to Tables in Lakehouse.

Hope this is helpful. Do let me know incase of further queries.

Re: dataflow

Anonymous — Fri, 17 May 2024 09:16:33 GMT

Hi Anonymous ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered.
Otherwise, will respond back with the more details and we will try to help .

Thanks

Re: dataflow

Anonymous — Mon, 20 May 2024 09:23:23 GMT