Forum Discussion
undefined
Hi,
we are using pileline copy activity to read data from xml into delta table.
In mappings tab, import schema is not working delta lake table. Is there any possible solution using copy activity? Thank you
Hi Sureshmannem ,
Thank you for the follow-up. We would be happy to assist you!
Upon my knowledge, this is a known limitation when using Copy Activity with Delta Lake tables in Fabric , the “Import Schema” feature is not supported, which explains the inconsistent behavior you are seeing with locked or missing column mappings. This is documented in the official guidance here: Copy data from/to Lakehouse – Limitations.
For more context, you may also refer to the link: Data Factory limitations in Fabric.
Hope this helps.
Thank you.
4 Replies
Hi Sureshmannem ,
Thank you for reaching out to us on Microsoft Fabric Community Forum!
This is a known limitation, Delta may need the schema to be set up ahead of time or mapped manually. The best workaround is to define the mappings yourself or make sure the table already exists with the right structure.
Refer the below solved thread and the document for more information:
https://community.fabric.microsoft.com/t5/Data-Engineering/Copy-data-activity-creates-no-proper-delta-table-with-schema-in/m-p/4387754
https://learn.microsoft.com/en-us/fabric/data-factory/connector-lakehouse-copy-activity#limitations
https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-databricks-delta-lake?tabs=data-factory
Hope this helps.If so,give us kudos and consider accepting it as solution.
Thank you.
Regards,
Pallavi.Hi Sureshmannem ,
I wanted to follow up to see if you had a chance to review the previous response provided by us. I hope it was helpful. If yes, please Accept the answer so that it will be helpful to others to find it quickly.
Thank you.Hi Anonymous ,
Thank you for your support and prompt follow-up.
Unfortunately, the current workaround isn’t resolving the issue. The copy activity against Delta Lake/Warehouse is not effectively managing column mappings.
- At times, the schema is recognized but doesn’t allow changes to the mappings.
- In other instances, the schema is ignored entirely, leaving blank fields.
I’m exploring alternative approaches using notebooks to overcome these limitations.
Best regards,
SureshHi Sureshmannem ,
Thank you for the follow-up. We would be happy to assist you!
Upon my knowledge, this is a known limitation when using Copy Activity with Delta Lake tables in Fabric , the “Import Schema” feature is not supported, which explains the inconsistent behavior you are seeing with locked or missing column mappings. This is documented in the official guidance here: Copy data from/to Lakehouse – Limitations.
For more context, you may also refer to the link: Data Factory limitations in Fabric.
Hope this helps.
Thank you.
