Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Enhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.

Reply
smpa01
Super User
Super User

Converting Fabric dataframe to spark dataframe

How can I convert a sempy.fabric.fabricdataframe to spark df?

The following does not work

 

dataset = (fabric
            .evaluate_dax(workspace= server,
            dataset=db,
            dax_string=query_string)
    ).to_pandas()


AttributeError: 'FabricDataFrame' object has no attribute 'to_pandas()'

 

 

Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
My custom visualization projects
Plotting Live Sound: Viz1
Beautiful News:Viz1, Viz2, Viz3
Visual Capitalist: Working Hrs
1 ACCEPTED SOLUTION
frithjof_v
Super User
Super User

 

dataset = (fabric
            .evaluate_dax(workspace= server,
            dataset=db,
            dax_string=query_string)
    )

spark_df = spark.createDataFrame(dataset)

 

 

This works for me.

 

I think you can treat the Fabric dataframe as a Pandas dataframe, and convert it to a Spark dataframe the same way you would convert a Pandas dataframe to a Spark dataframe. 

 

https://learn.microsoft.com/en-us/fabric/data-science/semantic-link-overview#fabricdataframe-data-st...

 

"
The FabricDataFrame class:

  • Supports all pandas operations.
  • Subclasses the pandas DataFrame and adds metadata, such as semantic information and lineage.
  • (...)

"

View solution in original post

2 REPLIES 2
frithjof_v
Super User
Super User

 

dataset = (fabric
            .evaluate_dax(workspace= server,
            dataset=db,
            dax_string=query_string)
    )

spark_df = spark.createDataFrame(dataset)

 

 

This works for me.

 

I think you can treat the Fabric dataframe as a Pandas dataframe, and convert it to a Spark dataframe the same way you would convert a Pandas dataframe to a Spark dataframe. 

 

https://learn.microsoft.com/en-us/fabric/data-science/semantic-link-overview#fabricdataframe-data-st...

 

"
The FabricDataFrame class:

  • Supports all pandas operations.
  • Subclasses the pandas DataFrame and adds metadata, such as semantic information and lineage.
  • (...)

"

Liked the 1-liner; took a longer route which I can happily discard

# data_as_dict = dataset.to_dict()
# columns = dataset.columns

# flattened_rows = [Row(**{col: data_as_dict[col][i] for col in columns}) for i in range(len(data_as_dict[columns[0]]))]

 

Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
My custom visualization projects
Plotting Live Sound: Viz1
Beautiful News:Viz1, Viz2, Viz3
Visual Capitalist: Working Hrs

Helpful resources

Announcements
July 2025 community update carousel

Fabric Community Update - July 2025

Find out what's new and trending in the Fabric community.

Join our Fabric User Panel

Join our Fabric User Panel

This is your chance to engage directly with the engineering team behind Fabric and Power BI. Share your experiences and shape the future.

June FBC25 Carousel

Fabric Monthly Update - June 2025

Check out the June 2025 Fabric update to learn about new features.