Forum Discussion
Issue with Dataflow Gen2 Copy
Hi NONO Chen,
Thank you for the response and the detailed suggestions.
To clarify further, no constraints are applied in this scenario, and there have been no changes to the data types, column names, or the number of columns between the source and target tables. Both tables are identical in structure.
Let me reiterate the key observation:
- Copy 1: Exists in the lakehouse with a schema applied.
- Copy 2: Exists in the lakehouse without a schema applied.
- It works successfully for Copy 2 but fails for Copy 1.
Based on this, I suspect the issue may not be related to column data types, names, or constraints.
Looking forward to your insights!
Best regards,
Billybilly
Hi,
unfortunately I don't have a solution to this problem, but I would just like to add that I'm having the exact same issue.
My table names also don't contain any special characters.
Example of table name is: companynamesalesdata
I was able to fix my issue, in the source data I had the source set to the lakehouse directly, which didn't work. I had to make a SQL connection to the SQL endpoint of the lakehouse and there my query worked without issues.
I hope this can help you too
Hi Anne-MarijnV,
Thank you for providing the solution. I used to extract data using the SQL endpoint and noticed that the Dataflow Gen2 keeps spinning for more than an hour. Normally, this process only takes a few minutes.
I wonder if you're experiencing the same issue on your end.
Best regards,
Billybilly