Forum Discussion
Add or remove column in Lakehouse table
- Anonymous2 years ago
Hi fabricator1
Thanks for using Fabric Community.
Currently ALTER TABLE command is not supported for tables in Lakehouse and Warehouse. So when you try to modify the table using Spark code it might result in some issues.
But as a work around I would suggest the following steps:1) I have a table named List in my lakehouse with one of the column named "genre".
2) Run the below given code by dropping the column required. Create a new table named list1.3) Delete the original list table from the lakehouse.
4) Now run the below code which will create a new table with name as List with the dropped column.5) This table can be accessed from the SQL endpoint also.
Hope this helps. Please let me know if you have any further questions. Glad to help. - 2 years ago
I didn't have the time to try this procedure. I started on it, and one thing I noticed was that when creating a table using Notebook, the table name can only have small letters. I guess that's normal.
I had created the original table using Dataflows Gen 2, and the table name had capital letters.
Anyway I decided to delete the table and recreate the table using Dataflows Gen 2. For my current purpose, that was okay.
I hope there will be a feature to add or remove columns in a Lakehouse table without having to delete and recreate the table. I created it as an idea: Microsoft Idea
I didn't have the time to try this procedure. I started on it, and one thing I noticed was that when creating a table using Notebook, the table name can only have small letters. I guess that's normal.
I had created the original table using Dataflows Gen 2, and the table name had capital letters.
Anyway I decided to delete the table and recreate the table using Dataflows Gen 2. For my current purpose, that was okay.
I hope there will be a feature to add or remove columns in a Lakehouse table without having to delete and recreate the table. I created it as an idea: Microsoft Idea
Hi fabricator1 , not sure if this helps with your particular scenrio but you could try the column mapping feature which allows you to alter/drop/add columns in a Lakehouse table by following this (it's databricks but it works in Fabric too). It's a one way operation on the table itself and can't be reversed.
Rename and drop columns with Delta Lake column mapping | Databricks on AWS
- fabricator12 years agoAdvocate II
Thanks AndyDDC,
This was my original approach, however after I did that I wasn't able to use the table in the SQL Analytics Endpoint and Power BI semantic model.
I guess that's what's mentioned here:
Is it working on your side after making such changes?- AndyDDC2 years agoMost Valuable Professional
Ah OK, no the meta-data sync will not work for the SQL Endpoint/Semantic Model unfortunately. I've used the above approach for a raw lakehouse layer (proof of concept) which did not require sql endpoint/semantic model