Forum Discussion
Fabric Notebook - write dataframe to lakehouse fails - 'DataFrame' object has no attri
Hi. I try to write some data to a lakehouse. Setup An lakehouse named "Metadata" A notebook Gets some data ❌Writes to lakehouse An error occurs when writing to the the lakehouse. Rep...
You definitely need to convert to a spark dataframe before writing. Can you show the code you used where you converted to a spark dataframe? I used spark.createDataFrame.
Hi. Thanks for the reply blopez11 .
It works using the convertion to spark dataframe as you suggested.
The code belows works:
import sempy
import sempy.fabric as fabric
workspaces = fabric.list_workspaces()
# For simplicity, keep columns withouth " " in column names in order to avoid Error Invalid column name when writing
workspaces_subset = workspaces[['Type', 'Id']]
workspaces_subset_spark = spark.createDataFrame(workspaces_subset)
# Write to deault lakehouse
table_name = "workspaces_subset"
workspaces_subset_spark.write.format("delta").mode("Overwrite").saveAsTable(table_name)
The error I experienced was probably since the dataframe was unintentionally converted to Series before trying to convert to spark dataframe.
The unintentionnaly convertion to Series was due to selecting 1 column from the dataframe, i.e.
# Data type unintenionally change from pandas dataframe to Series
workspaces_series = workspaces_df['Id']
Thanks for the help!