topic Re: Data pipeline COPY activity vs Dataflow Gen2 text ingest with lowercase in Data Engineering

Data pipeline COPY activity vs Dataflow Gen2 text ingest with lowercase

amaaiia — Wed, 17 Apr 2024 08:26:31 GMT

I've ingested one table with DFg2 and the same table with COPY activity. I've realised that one column is in uppercase when DFg2 is used, and in lowercase when COPY is used. In source (SQL Server on prem) is in uppercase. Why is COPY activity parsing from uppercase to lowercase?

This is the UNION of the same row in both tables:

Re: Data pipeline COPY activity vs Dataflow Gen2 text ingest with lowercase

Anonymous — Thu, 18 Apr 2024 09:34:09 GMT

Hi amaaiia
Thanks for using Fabric Community.
I tried to repro the above scenario, but did not face any issue as mentioned above. Are you facing this issue with all the tables and columns?
Please try again and let me know if the issue persists.
Thanks

Re: Data pipeline COPY activity vs Dataflow Gen2 text ingest with lowercase

Anonymous — Fri, 19 Apr 2024 17:26:34 GMT

Hi amaaiia
We haven’t heard from you on the last response and was just checking back to see if your query got resolved. Otherwise, will respond back with the more details and we will try to help.
Thanks

Re: Data pipeline COPY activity vs Dataflow Gen2 text ingest with lowercase

amaaiia — Mon, 22 Apr 2024 14:07:01 GMT

No. I've checked the query executed at source. If I run this query directly at source I get the results in uppercase. However, when running from COPY activity, I get it in lowercase.

Re: Data pipeline COPY activity vs Dataflow Gen2 text ingest with lowercase

Anonymous — Mon, 22 Apr 2024 17:51:08 GMT

Hi amaaiia
Apologies for the issue you have been facing. The internal team has confirmed that this is a bug. I have created a bug for the same. The team is working on this to fix the issue. I will keep you posted regarding the updates.
Appreciate your patience.