Overwrite of SQL Server table into lakehouse fails

[amaaiia]

Hi,

I have a Data Pipeline in Production that COPIES SQL Server tables into Lakehouse tables. The Pipeline does an overwrite of each table every time. All tables work well except one.

1. I count rows with preview:

2. I configure source and I preview data correctly:

3. I configure destination:

4. Run pipeline and it fails:

I've done some testing:

• If I run a bit different query that returns the same number of records, it works:
  
• This query also fails:

• This query also fails:

If I preview the data it works, but if I run the Pipeline it fails, always the same error. The only way I've seen it works, is when I filter the query with one filed value. I don't think is a column names issue because the query that is working selects all the columns.

[v-karpurapud]

Hi @amaaiia 

We are following up to see if your query has been resolved. Should you have identified a solution, we kindly request you to share it with the community to assist others facing similar issues.

If our response was helpful, please mark it as the accepted solution and give a kudos, as this helps other members in  community.

Thank you

[amaaiia · Accepted solution]

Finally, I ask SQL Server admin to create the same table in source in a different way. We don't know which was the reason of the failure but changing the table create mode, now it's working.

[ahmetyilmaz]

Try doing your own mapping manually with + New mapping. You already have access to column names and data types.

[amaaiia]

I ingest some tables inside a loop. I can't specify the schema because it's different for each table. I use the same COPY activity for all the tables, setting table_name as a variable in de COPY activity.

[ahmetyilmaz]

Can you try what I said just to test the problem? At least we can determine which column is the problem.

[ahmetyilmaz]

Hello, have you tried selecting the import schemas option from the mapping section?
If there is an empty column, maybe you can see it here.

[amaaiia]

Unable to import schema. I click on Import schemas, it says Loading but with no error message, it doesn't import it.