topic Re: Loading delta tables into a lakehouse in Data Engineering

Loading delta tables into a lakehouse

Anonymous — Wed, 15 May 2024 08:16:35 GMT

Hi All,

Please HELP, I am new to fabric and struggling to load my files to delta tables in a lakehouse.

once uploading my data file (in CSV.) format and then Load to tables it gives an error regarding my CSV headers see below.

I don't know what else to do as it allows me to load delta tables without the headers. Please help

Server Error.
Message: Invalid column name(s) 'OnBDQ;Onshortregimen;RegimenType;RegistrationYear;Type;DRTBUnit;DRTBProvince;DRTBSubDistrict;DRTBDistrict;PreviousDrug;Gender;TreatmentOutcome' in file 'Files/Sedibengdata2.csv'. Column names must contain UTF-8 encoded Unicode word characters and can be a maximum of 128 characters long. Unicode word characters include letters of any case and their nonspacing marks, punctuation connectors like the underscore(_), and decimal digits, but no space characters are allowed.
Activity Id (Failed Request) : 23167694-e0ee-47d6-b79f-d85d20abbce1
Error Code : InvalidColumnName
HTTP response code : 400
User session Id : 696e2c1f-1ad3-4590-b9c8-485da94199d0

Re: Loading delta tables into a lakehouse

v-cboorla-msft — Wed, 15 May 2024 09:15:32 GMT

Hi Anonymous

Thanks for using Microsoft Fabric Community.

The error message you received indicates that there are invalid column names in your CSV file. Specifically, the column names listed include characters that do not meet the requirements for Delta tables.

Delta tables expect column names to be UTF-8 encoded Unicode word characters, which means they can include letters (of any case), nonspacing marks, punctuation connectors (like underscores), and decimal digits. However, spaces are not allowed. To resolve this, ensure that your CSV file’s column headers adhere to these rules.

Table names must contain alphanumeric characters and underscores. Column names allow any English letters (upper or lower case), underscores, and characters from other languages (such as Chinese) in UTF, up to 32 characters. If a proper column name cannot be achieved during validation, the load action will fail.

I hope this information helps. Please do let us know if you have any further queries.

Thank you.

Re: Loading delta tables into a lakehouse

Anonymous — Wed, 15 May 2024 09:54:38 GMT

Thanks for the help, how do I ensure that my headers are UTF-8 encoded Unicode word characters? I am so lost, thank you for the explanation above.

Re: Loading delta tables into a lakehouse

MartinMason — Sat, 18 May 2024 22:07:18 GMT

I'm pretty sure your issue is that your column names contain spaces, i.e. Customer Name. You'll need to go through the list of columns and replace the spaces, either CustomerName or Customer_Name are both valid.

Re: Loading delta tables into a lakehouse

BartStens — Wed, 22 May 2024 13:20:55 GMT

Dear Anonymous ,

I was also struggling with exact same error msg. My headers followed requirements exactly as UTF-8 demands, but still got the error msg.

Turns out I had to specify my column-seperator a.k.a. delimiter in the prompt when selecting the three dots near the file > "Load file in new table" then enter de correct delimiter used in your .csv-file, in my case a comma (,) was pre-filled in the prompt, but my .csv-file used (;) so had to adjust that one and load was succesfull!

See below screencap (sorry it's in Dutch-UI)

Re: Loading delta tables into a lakehouse

swapnasam — Thu, 19 Jun 2025 00:43:57 GMT

Hi, I had the same problem and its just when you try to load the dataset to Delta Table from UI, its forcing us to use a proper format for column headers with no spaces or special character.

You can solve this problem by using a notebook

Load the csv into a pandas df and write it with delta.columnMapping.mode", "name" which lets the df to write into a Table with spaces in column headers.

```

import pandas as pd

# Load data into pandas DataFrame from "/lakehouse/default/Files/metadata.csv"

df = pd.read_csv("/lakehouse/default/Files/metadata.csv")

display(df)

spark_df = spark.createDataFrame(df)

spark_df.write.format("delta").option("delta.columnMapping.mode", "name").saveAsTable("dbo.Sample")

```

Hope this helps.

Thanks