Forum Discussion
Gen2 df external table
At the moment, Dataflow Gen2 only loads data to tables. Do feel free to suggest new destinations (and formats) in the Fabric Ideas portal (https://aka.ms/FabricIdeas)
An alternative is to leverage the copy activity or a copy job. Especially as the bronze layer is typically used for the files in its raw state, so no transformation should be performed at that layer and instead a simple copy activity should be good enough. If a connector is missing from the copy job / copy activity, then would you mind letting us know what the source is? you can also post a new idea for such connector in the Ideas Portal.
Hi smpa01
You arr right,Dataflow Gen2 currently supports writing data only to Lakehouse tables, not specific subfolders.
one thing which you can try is Use Dataflow Gen2 to land data in a staging table and then read this staging table in notebook and write to your desired location
That's too much honestly to maintain. pre_bronze->brone->silver so on and so forth.
Dataflows have an advantage over notebooks when it comes to connecting to certain sources that don't have equivalent connectors available in notebooks — for example, on-premises SQL Server, SharePoint, etc. In such cases, there is no alternative but to use a dataflow.
Currently, dataflows remain relevant largely because of this limitation. So, for writing to destination, it only makes sense that df gen2 provides same options as notebook.
To keep up with the norm, df gen2 must give the ability to write to subfolders. After all, any bronze should land in files for audit trailing.
