Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Enhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.

Reply
smpa01
Super User
Super User

Converting Fabric dataframe to spark dataframe

How can I convert a sempy.fabric.fabricdataframe to spark df?

The following does not work

 

dataset = (fabric
            .evaluate_dax(workspace= server,
            dataset=db,
            dax_string=query_string)
    ).to_pandas()


AttributeError: 'FabricDataFrame' object has no attribute 'to_pandas()'

 

 

Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
My custom visualization projects
Plotting Live Sound: Viz1
Beautiful News:Viz1, Viz2, Viz3
Visual Capitalist: Working Hrs
1 ACCEPTED SOLUTION
frithjof_v
Super User
Super User

 

dataset = (fabric
            .evaluate_dax(workspace= server,
            dataset=db,
            dax_string=query_string)
    )

spark_df = spark.createDataFrame(dataset)

 

 

This works for me.

 

I think you can treat the Fabric dataframe as a Pandas dataframe, and convert it to a Spark dataframe the same way you would convert a Pandas dataframe to a Spark dataframe. 

 

https://learn.microsoft.com/en-us/fabric/data-science/semantic-link-overview#fabricdataframe-data-st...

 

"
The FabricDataFrame class:

  • Supports all pandas operations.
  • Subclasses the pandas DataFrame and adds metadata, such as semantic information and lineage.
  • (...)

"

View solution in original post

2 REPLIES 2
frithjof_v
Super User
Super User

 

dataset = (fabric
            .evaluate_dax(workspace= server,
            dataset=db,
            dax_string=query_string)
    )

spark_df = spark.createDataFrame(dataset)

 

 

This works for me.

 

I think you can treat the Fabric dataframe as a Pandas dataframe, and convert it to a Spark dataframe the same way you would convert a Pandas dataframe to a Spark dataframe. 

 

https://learn.microsoft.com/en-us/fabric/data-science/semantic-link-overview#fabricdataframe-data-st...

 

"
The FabricDataFrame class:

  • Supports all pandas operations.
  • Subclasses the pandas DataFrame and adds metadata, such as semantic information and lineage.
  • (...)

"

Liked the 1-liner; took a longer route which I can happily discard

# data_as_dict = dataset.to_dict()
# columns = dataset.columns

# flattened_rows = [Row(**{col: data_as_dict[col][i] for col in columns}) for i in range(len(data_as_dict[columns[0]]))]

 

Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
My custom visualization projects
Plotting Live Sound: Viz1
Beautiful News:Viz1, Viz2, Viz3
Visual Capitalist: Working Hrs

Helpful resources

Announcements
July 2025 community update carousel

Fabric Community Update - July 2025

Find out what's new and trending in the Fabric community.

June FBC25 Carousel

Fabric Monthly Update - June 2025

Check out the June 2025 Fabric update to learn about new features.