Forum Discussion
Helper IICopy Data to Lakehouse - Edit Columns
Hi All,
I am copying data from a on premise SQL data source with a series of copy-assistant 'for each loops' to a lakehouse. However one of loops is failing due to a space in a column destination name.
Problem is I am trying to find a way to edit the columns without deleting the whole thing and I can't find anyway. Is this possible?
Thanks
Hello ringovski
If your column header having spaces, your lakehosue won't allow to ,load the csv file into destination. You can read this blog Optimizing CSV Ingestion: Transforming Space-Delim... - Microsoft Fabric Community It might help you to resolve your problem
Thank you!!!
Did I answer your question? Mark my post as a solution!
Proud to be a Super User!Hi ringovski ,
Thank you for reaching out to the Microsoft Fabric Community and providing the details.
After reviewing your setup and the conversation, I can confirm that suparnababu8 gave the right advice. The issue happens because Lakehouse doesn’t support column names with spaces, which is why your pipeline fails.
The best way to fix this without rebuilding the whole pipeline is to add a Dataflow Gen2 activity before the Copy Data activity. This lets you pull the data from SQL, rename columns to remove spaces (e.g., Customer Name to Customer_Name), and send the clean data to your Lakehouse. Once the columns are cleaned up in Dataflow Gen2, the Copy Data activity should work without errors.
You can find detailed steps on setting up Dataflow Gen2 here: Create your first Dataflow Gen2 in Microsoft Fabric.
I hope my suggestions give you good idea, if you need any further assistance, feel free to reach out.
If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.
Thank you.
9 Replies
Hi ringovski ,
Thank you for reaching out to the Microsoft Fabric Community and providing the details.
After reviewing your setup and the conversation, I can confirm that suparnababu8 gave the right advice. The issue happens because Lakehouse doesn’t support column names with spaces, which is why your pipeline fails.
The best way to fix this without rebuilding the whole pipeline is to add a Dataflow Gen2 activity before the Copy Data activity. This lets you pull the data from SQL, rename columns to remove spaces (e.g., Customer Name to Customer_Name), and send the clean data to your Lakehouse. Once the columns are cleaned up in Dataflow Gen2, the Copy Data activity should work without errors.
You can find detailed steps on setting up Dataflow Gen2 here: Create your first Dataflow Gen2 in Microsoft Fabric.
I hope my suggestions give you good idea, if you need any further assistance, feel free to reach out.
If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.
Thank you.
Hi ringovski ,
I wanted to follow up on our previous suggestions regarding the issue. We would love to hear back from you to ensure we can assist you further.
If my response has addressed your query, please accept it as a solution and give a ‘Kudos’ so other members can easily find it. Please let us know if there’s anything else we can do to help.
Thank you.
Hi ringovski ,
I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.
Thank you.
- suparnababu8
Super User
Hello ringovski
If your column header having spaces, your lakehosue won't allow to ,load the csv file into destination. You can read this blog Optimizing CSV Ingestion: Transforming Space-Delim... - Microsoft Fabric Community It might help you to resolve your problem
Thank you!!!
Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
ringovskiThanks for the reply however this is not my issue and doesn't resolve my problem.
- suparnababu8
Hi ringovski
You can copy your files in this way, On-premises-->Dataflows-->Lakehouse (By using pipelines). Bcz if you have spaces in your column headers it will not sucess. Just give a try in this way.
Thank you!!!
Did I answer your question? Mark my post as a solution!
Proud to be a Super User!