Forum Discussion

Billybilly's avatar
Billybilly
Frequent Visitor
1 year ago

Issue with Dataflow Gen2 Copy

I am experiencing an issue with copying two tables from one lakehouse (with schema) to another lakehouse. The copy operation fails, but it succeeds when copying without the schema (using the same tab...
Dataflow
Need Help
Anne-MarijnV's avatar
Anne-MarijnV
Frequent Visitor
1 year ago

Hi,

 

unfortunately I don't have a solution to this problem, but I would just like to add that I'm having the exact same issue.

My table names also don't contain any special characters.

Example of table name is: companynamesalesdata

Anne-MarijnV's avatar
Anne-MarijnV
Frequent Visitor
1 year ago

I was able to fix my issue, in the source data I had the source set to the lakehouse directly, which didn't work. I had to make a SQL connection to the SQL endpoint of the lakehouse and there my query worked without issues.

I hope this can help you too

  • Billybilly's avatar
    Billybilly
    Frequent Visitor
    1 year ago

    Hi Anne-MarijnV,

    Thank you for providing the solution. I used to extract data using the SQL endpoint and noticed that the Dataflow Gen2 keeps spinning for more than an hour. Normally, this process only takes a few minutes.

    I wonder if you're experiencing the same issue on your end.

    Best regards,

    Billybilly