topic Re: Use a Data Pipeline to COPY some tables in a lakehouse into another lakehouse in Pipelines

Use a Data Pipeline to COPY some tables in a lakehouse into another lakehouse

amaaiia — Fri, 14 Feb 2025 08:31:18 GMT

Hi.

I have a lakehouse with several tables that are partitioned by 5 fields (type, frequency, YEAR, MONTH, DAY). I want to make a Data Pipeline that copies the tables to another lakehouse. I have prepared the COPY activity like this, and I try to preview the data of one table:

I get this error:

I can query the tables in SQL Enpoint, in the notebook.. with no problems. But I can't preview nor copy the tables. Any ideas?

Re: Use a Data Pipeline to COPY some tables in a lakehouse into another lakehouse

Anonymous — Fri, 14 Feb 2025 09:47:36 GMT

Hi amaaiia ,

According to the error message provided in the image, 20065 is usually caused by inconsistent partition path patterns or metadata issues. You can try checking the partition path, repairing the data files, updating the metadata and adjusting the COPY active configuration.

Best Regards,
Adamk Kong

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Re: Use a Data Pipeline to COPY some tables in a lakehouse into another lakehouse

amaaiia — Fri, 14 Feb 2025 12:36:29 GMT

I don't know what can be wrong. I've done some testing:

1. Read the original table and write it as new table with the same partitioning fields:

2. Go to pipeline, select the new table and try to preview it:

Same error. Is a new table, partitioned by 5 fields. What can be wrong?