Forum Discussion
Solved
Importing CSV merge error
I have a script that uploads a daily CSV to Fabric, then imports it into a table. There's one column that's a doulbe data type that I consistently have issues with. Sometimes the CSV will accurately ...
Hi wowcatfactor,
How are you importing the CSV? Here's some recommendations based on various ingestion methods into Fabric:
- Dataflow Gen2 (Power Query): In the last steps, explicitly set the column to Fixed decimal number (currency) or Decimal number with Table.TransformColumnTypes, then load to the table. (Power Query data types, Dataflow Gen2 destinations)
- Copy Activity (Pipeline): In Mappings, set the sink type to decimal(p,s) (e.g., decimal(18,6)) for that column. (Fabric Data Factory type mapping)
- Spark/Notebook: Read the CSV with an explicit schema (DoubleType or DecimalType) and write to Delta. (Delta schema enforcement)
If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.
Hey wowcatfactor ,
This is a typical CSV problem that occurs repeatedly.
Unfortunately, you have to intercept these things beforehand.
I would do exactly what tayloramy suggests, and it's a good approach to solving your problem.
Best regards