Forum Discussion
Advocate IISynapse Datawarehouse to Fabric Warehouse Data Load
Well we have tested this scenerio by creating tabular mapping and directly loading parquet file from data lake into the lakehouse using ADF and then moving that data into warehouse through stored procedure works; without any staging and it works, somehow staging in onelake does not allow us to move the data into warehouse.
This will be an actual solution for the problemetic data types, or someone who is directly looking to copy data to test fabric warehouse/lakehouse performance and test one time migration.
Community SupportThanks for your response. Based on Microsoft documented behavior, this issue stems from how Fabric pipelines automatically stage data in OneLake using Parquet format when loading from Synapse to a Fabric Warehouse. During this process, FLOAT values are internally converted to DECIMAL(22,8) with a FIXED_LEN_BYTE_ARRAY, physical type, which causes a mismatch when inserting into a target column defined as FlOAT.
Reference:
Troubleshoot the Parquet format connector - Microsoft Fabric | Microsoft Learn
Could you please try the syntax AntoineW suggested and let me know how it behaves in your setup.
Advocate IIIt wont work and most of the values are NULL for us, I think we need to load it from lakehouse to override the step of not using the staging and directly copy the parquet file with tabular mapping and enabling float datatypes.
- V-yubandi-msft
Hi AnmolGan81 ,
Thank you for sharing your findings. Loading data directly from the Lakehouse and bypassing the automatic staging step seems like a good approach, especially if tabular mapping preserves the FLOAT data type. Considering the staging limitations, this could serve as an effective workaround.I look forward to hearing about your results after you try it.
Thank You.- AnmolGan81
we have an upcoming migration to other enviorment will let you know how it goes.
- V-yubandi-msft
Thank you for the update. I hope the migration to the new environment goes smoothly. When you've had a chance to test everything, please let me know how it works out.