cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
marcosvin
Resolver I
Resolver I

What is the best way to load a fixed-size CSV file to the Warehouse (Microsoft Fabric)?

I did a test of using Dataflow Gen2 and was able to read the CSV file by position, but I can only upload to Lakehouse (not Warehouse). In the test using the data pipelines I did not find the option to read by position. Can you tell if there is such an option? Or thinking about the architecture the option of loading first to Lakehouse is correct?

1 ACCEPTED SOLUTION
marcosvin
Resolver I
Resolver I

4 REPLIES 4
marcosvin
Resolver I
Resolver I

I found the solution to use Dataflow Gen2 and warehouse in this topic:

Solved: Dataflow (Gen2): Fabric Warehouse Destination...th... - Microsoft Fabric Community

makromer
Microsoft
Microsoft

In Fabric Data Pipelines, you can use the Copy task to bring data directly into the Warehouse. Create a new data pipeline, click on "Copy Data" and select your CSV file. Currently in pipelines, your file must be in Azure, but we are adding access to on-prem data as well. Then you can select "Data Warehouse" as your target. It is in the Workspace category for data destinations.

MartinEstZurita
New Member

Hi marcosvin, as far as I know it is not required to upload first to lakehouse and then to warehouse. 

 

I dont understand what you mean with "read by position", could you clarify that?

Hi! The CSV file is delimited by position, that is, the first column starts at character 1 and goes up to 10, the second from 11 to 20, and so on... I didn't find the functionality to read the file this way.

Helpful resources

Announcements
Fabric August Update Carousel

Fabric August 2023 Update

Check out the August 2023 Fabric update to learn about new features.

Data Factory Founders Badge

Data Factory Community Founders Badge

Become a founding member of the Data Factory Community.

Learn Live

Learn Live: Event Series

Join Microsoft Reactor and learn from developers.

Data Factory Webinar Series Sept

Join the Data Factory Webinar series!

Want to learn more about Data Factory in Fabric? Join our webinars this month to build up your Data Factory skills!

Top Solution Authors