Forum Discussion
Decimal precision exceeds when copying data from SQL Server to Databricks using Fabric Copy activity
- 7 months ago
Hi renjith_kumar , so it's a two-questions, let's go one at a time
#1. Why is the decimal precision increasing during the copy? During Fabric Copy using the Spark ODBC driver, Spark does not preserve the original precision during intermediate operations and applies expression widening rules (e.g. when decimals are cast implicitly, moved through an ODBC layer, passed through serialization, etc.), furhermore Fabric Copy also does schema reconciliation and attempts type safety forcing Spark to materialize the entire schema upfront, which triggers decimal widening.
#2. how can I prevent this while copying data from SQL Server to Databricks using Fabric? You can...
a) Explicit CAST in Source Query (Recommended) in a view and use this as the source query in Fabric Copy, not the table.
b) Use a use a pre‑created table as a target and configure Copy Activity to Disable auto‑create and schema drift
Thanks for being so precise in the description of your issue as this makes the analysis easier, hope you find this information useful and you get your pipeline working as expected... a would appreciate a kudos and if you consider, accept this as solution. All the very best!
Hi renjith_kumar,
I hope the information provided above assists you in resolving the issue. If you have any additional questions or concerns, please do not hesitate to contact us. We are here to support you and will be happy to help with any further assistance you may need.
Regards,
Community Support Team.
Hi renjith_kumar,
I hope the above details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We are always here to support you.
Regards,
Community Support Team.