Forum Discussion
Solved
Importing CSV merge error
I have a script that uploads a daily CSV to Fabric, then imports it into a table. There's one column that's a doulbe data type that I consistently have issues with. Sometimes the CSV will accurately ...
Hi wowcatfactor,
How are you importing the CSV? Here's some recommendations based on various ingestion methods into Fabric:
- Dataflow Gen2 (Power Query): In the last steps, explicitly set the column to Fixed decimal number (currency) or Decimal number with Table.TransformColumnTypes, then load to the table. (Power Query data types, Dataflow Gen2 destinations)
- Copy Activity (Pipeline): In Mappings, set the sink type to decimal(p,s) (e.g., decimal(18,6)) for that column. (Fabric Data Factory type mapping)
- Spark/Notebook: Read the CSV with an explicit schema (DoubleType or DecimalType) and write to Delta. (Delta schema enforcement)
If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.
Hi wowcatfactor,
How are you importing the CSV? Here's some recommendations based on various ingestion methods into Fabric:
- Dataflow Gen2 (Power Query): In the last steps, explicitly set the column to Fixed decimal number (currency) or Decimal number with Table.TransformColumnTypes, then load to the table. (Power Query data types, Dataflow Gen2 destinations)
- Copy Activity (Pipeline): In Mappings, set the sink type to decimal(p,s) (e.g., decimal(18,6)) for that column. (Fabric Data Factory type mapping)
- Spark/Notebook: Read the CSV with an explicit schema (DoubleType or DecimalType) and write to Delta. (Delta schema enforcement)
If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.