Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Try your skills in the Power BI Dataviz World Championship! Round one ends June 26. Join now

Reply
syscpd
Regular Visitor

Import Data from PySpark to Table

Hi everyone,

We are working on Fabric Preview and today we are trying to import data from DataFrame in NoteBook (PySpark) into a Table in Lakehouse.

Here is the code we have used:

 

table_name = "dds_estimated"
spark_df.write.mode("overwrite").format("delta").option("mergeSchema", "true").save(f"Tables/{table_name}")
print(f"Done loading {table_name}")

 

 

Then we encountered a problem in the table format and were asked to use the following command to edit the table properties:

 

spark.sql(f'''
    ALTER TABLE dds_estimated SET TBLPROPERTIES (
       'delta.columnMapping.mode' = 'name',
       'delta.minReaderVersion' = '2',
       'delta.minWriterVersion' = '5'
    )
''')

 

 

Overall, we can run the notebook done but nothing import to our table, it's still blank and have a notice:
Table uses column mapping which is not supported.

 

What should we do to import DataFrame to Table in Lakehouse?

0 REPLIES 0

Helpful resources

Announcements
Fabric Data Days is here Carousel

Fabric Data Days 2026

Don't miss out on Data Days, June 15 through August 7. Learn Fabric, Power BI, SQL, AI and more.

May Power BI Update Carousel

Power BI Monthly Update - May 2026

Check out the May 2026 Power BI update to learn about new features.

Power BI DataViz World Championships carousel

Power BI DataViz World Championships - June 2026

A new Power BI DataViz World Championship is coming this June! Don't miss out on submitting your entry.