Check your eligibility for this 50% exam voucher offer and join us for free live learning sessions to get prepared for Exam DP-700.
Get StartedDon't miss out! 2025 Microsoft Fabric Community Conference, March 31 - April 2, Las Vegas, Nevada. Use code MSCUST for a $150 discount. Prices go up February 11th. Register now.
Situation:
i struggle to load a parquet file to DWH.
The Parquet File has a column "Info" which is nvarchar(max)
FabricDWH does only support a max of varchar(8000)
I would be fine if the column-data gets truncated on import to DWH so that only first 8000 characters are stored in DWH
Problem:
If i create a Fabric Pipeline with Copy Activity with/without Column Mapping i get:
ErrorCode=UserErrorSqlDWCopyCommandError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=SQL DW Copy Command operation failed with error 'String or binary data would be truncated while reading column of type 'VARCHAR(8000)
There are no options in PipelineUI Sink or Mapping Tab to define a convert or "allow truncate" o explicit type casts
so i tried to define "dynamic content" with a TabularTanslator in Mapping Tab of Copy Activity:
{"type": "TabularTranslator"
,"typeConversion": true
,"typeConversionSettings": {"allowDataTruncation": true
,"treatBooleanAsNumber": false}
}
This worked in AzureADF without issues, but when using in Fabric Pipeline i get error:
ErrorCode=UnsupportCopyFeatureInExternalCommand,The copy feature: TypeConversion is not supported by the external command SynapseImportCommand
Now the questions:
Solved! Go to Solution.
Hi @jochenj,
Thanks for using the Fabric community,
Data truncate is not a supported feature in DWH. Thus Data pipeline cannot support truncate for Parquet -> DWH. However Data pipeline can support skip incompatibly rows and output incompatibly rows by 'Settings'->'Fault Tolerance'.
Hope this is helpful. Incase of any further queries do let us know.
Hi @jochenj,
Thanks for using the Fabric community,
Data truncate is not a supported feature in DWH. Thus Data pipeline cannot support truncate for Parquet -> DWH. However Data pipeline can support skip incompatibly rows and output incompatibly rows by 'Settings'->'Fault Tolerance'.
Hope this is helpful. Incase of any further queries do let us know.
thanks for the insights. Can you confirm that even "renaming" of columns with a dynamic TabularTranslator expression should work? Or is this also not implemented yet?
Hello @jochenj ,
We haven’t heard from you on the last response and was just checking back to see if your query got resolved? Otherwise, will respond back with the more details and we will try to help.
Hi @jochenj ,
Glad to know your query got resolved. Please continue using Fabric Community incase of any queries.
User | Count |
---|---|
8 | |
3 | |
2 | |
2 | |
2 |
User | Count |
---|---|
12 | |
7 | |
5 | |
3 | |
3 |