Forum Discussion

AnmolGan81's avatar
AnmolGan81
Icon for Advocate II rankAdvocate II
1 year ago
Solved

Synapse Datawarehouse to Fabric Warehouse Data Load

We are trying to load data from synapse to fabric but getting one issue that is related to float data type that we have in target, during the load it is staging the data into one lake, and not able t...
data engineering
  • AnmolGan81's avatar
    AnmolGan81
    11 months ago

    Well we have tested this scenerio by creating tabular mapping and directly loading parquet file from data lake into the lakehouse using ADF and then moving that data into warehouse through stored procedure works; without any staging and it works, somehow staging in onelake does not allow us to move the data into warehouse.

    This will be an actual solution for the problemetic data types, or someone who is directly looking to copy data to test fabric warehouse/lakehouse performance and test one time migration.

BalajiL's avatar
BalajiL
Icon for Resolver I rankResolver I
1 year ago

hi AnmolGan81 

Float is being mapped as Decimal(22,8) in the staging parquet file. This cause the pipeline failure. 

1. Can you change the target table schema datatype to Decimal(22,8)  or Double and try it. 

or 

2. Cast datatype in your source query when writing to staging layer 

SELECT CAST(my_float_column AS DECIMAL(22,8)) AS my_float_column