Forum Discussion
Fabric Data Flow Gen2 Destination as Lakehouse Files
Hi,
I have tried to load the data into Microsoft Fabric Lakehouse as files using Data Flow Gen2, but there is no option to select as files. It default to load as tables into Lakehouse using Data Flow Gen2.
I can able to see the options in Copy Activity to select either File or Table.
Since many source connectors (Example : Salesforce) are not available in Copy Activity, so we have to dependent on Data Flow Gen2.
Any idea to load as files into Lakehouse using Data Flow Gen2 or it is limitation of Data Flow Gen2.
COPY Activity :
DataFlow Gen2 :
Regards,
Sathish Kumar
You could use Notebooks as well in Fabric. Currently Dataflow Gen2 doesn't support the load operation as files to a destination, but you can request that as new feature idea from aka.ms/FabricIdeas
You could alternatively load the data first as a table and then use something like Data Pipeline to copy the data and then save it as a file using the "Copy" activity. You could also take advantage of the "Copy" activity in pipelines and the REST API for it, or raise a new idea to have new explicit connectors to a source of your choice that might not be available today in Data Pipeline.
The most explicit way would be to use a Notebook.
Hope this helps!
6 Replies
Hi Anonymous ,
Apologize for the confusion. You're right, Dataflow Gen2 currently has limitations when it comes to directly loading data as files into the Fabric Lakehouse. Dataflow Gen2 focuses on transforming and loading data into tables within the Lakehouse.
Appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.
Hope this is helpful. Please let me know incase of further queries.