Forum Discussion
Fabric Data Flow Gen2 Destination as Lakehouse Files
You could use Notebooks as well in Fabric. Currently Dataflow Gen2 doesn't support the load operation as files to a destination, but you can request that as new feature idea from aka.ms/FabricIdeas
You could alternatively load the data first as a table and then use something like Data Pipeline to copy the data and then save it as a file using the "Copy" activity. You could also take advantage of the "Copy" activity in pipelines and the REST API for it, or raise a new idea to have new explicit connectors to a source of your choice that might not be available today in Data Pipeline.
The most explicit way would be to use a Notebook.
Hope this helps!
Hi Anonymous ,
Apologize for the confusion. You're right, Dataflow Gen2 currently has limitations when it comes to directly loading data as files into the Fabric Lakehouse. Dataflow Gen2 focuses on transforming and loading data into tables within the Lakehouse.
Appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.
Hope this is helpful. Please let me know incase of further queries.
Thanks for the confirmation, as it is Data Flow Gen2 limitation.
This case how we can extract the data from Salesforce and ingest as files into Lakehouse using Fabric.
Regards,
Sathish Kumar
You could use Notebooks as well in Fabric. Currently Dataflow Gen2 doesn't support the load operation as files to a destination, but you can request that as new feature idea from aka.ms/FabricIdeas
You could alternatively load the data first as a table and then use something like Data Pipeline to copy the data and then save it as a file using the "Copy" activity. You could also take advantage of the "Copy" activity in pipelines and the REST API for it, or raise a new idea to have new explicit connectors to a source of your choice that might not be available today in Data Pipeline.
The most explicit way would be to use a Notebook.
Hope this helps!
Hi Anonymous ,
If you are looking for options to load them as files in lakehouse. At present the alternative way is to use Azure Data Factory and can load data from Salesforce to Fabric Lakehouse.
Docs to refer -
Copy data from and to Salesforce - Azure Data Factory & Azure Synapse | Microsoft Learn
Copy and transform data in Microsoft Fabric Lakehouse - Azure Data Factory & Azure Synapse | Microsoft Learn
Hope this is helpful. Do let me know incase of further queries.