Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The Power BI Data Visualization World Championships is back! Get ahead of the game and start preparing now! Learn more

Reply
syscpd
Regular Visitor

Import Data from PySpark to Table

Hi everyone,

We are working on Fabric Preview and today we are trying to import data from DataFrame in NoteBook (PySpark) into a Table in Lakehouse.

Here is the code we have used:

 

table_name = "dds_estimated"
spark_df.write.mode("overwrite").format("delta").option("mergeSchema", "true").save(f"Tables/{table_name}")
print(f"Done loading {table_name}")

 

 

Then we encountered a problem in the table format and were asked to use the following command to edit the table properties:

 

spark.sql(f'''
    ALTER TABLE dds_estimated SET TBLPROPERTIES (
       'delta.columnMapping.mode' = 'name',
       'delta.minReaderVersion' = '2',
       'delta.minWriterVersion' = '5'
    )
''')

 

 

Overall, we can run the notebook done but nothing import to our table, it's still blank and have a notice:
Table uses column mapping which is not supported.

 

What should we do to import DataFrame to Table in Lakehouse?

0 REPLIES 0

Helpful resources

Announcements
Power BI DataViz World Championships

Power BI Dataviz World Championships

The Power BI Data Visualization World Championships is back! Get ahead of the game and start preparing now!

December 2025 Power BI Update Carousel

Power BI Monthly Update - December 2025

Check out the December 2025 Power BI Holiday Recap!

FabCon Atlanta 2026 carousel

FabCon Atlanta 2026

Join us at FabCon Atlanta, March 16-20, for the ultimate Fabric, Power BI, AI and SQL community-led event. Save $200 with code FABCOMM.