Managing Data Type Conversions for Silver Data Pipeline

Hi Anonymous ,

Few ways to handle type conversion,

Notebook:

1. Have the mapping sheet for the datatypes between pyspark and fabric. 

2. Create table with the proper datatype required for Fabric

3. Define your schema and read the parquet files. Use casting only on the required places. or Use infer schema and read the parquet files.

Copy Activity:

1. If no transformation required then you can use copy activity and under mapping you can see Type Conversion settingsfor datetime data type .

2. For other types if you want to convert use import schema and change the data type for destination table.

Regards,

Srisakthi