Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Enhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.

Reply
dgwilson
Resolver III
Resolver III

NoteBook and delta table write/read question

Hi All.

 

I'm unsure if this is the right place to ask my question, so apologies if it is not.

 

Using PySpark

I've got some code that writes a dataframe to a delta table in an attached lakehouse.

 

spark_df.write.format("delta") \
    .mode("overwrite") \
    .saveAsTable(delta_table_name)

 

 

After... I try and read that data via

 

read_df = http://spark.read.format("delta").table(delta_table_name)

 

And I get some weird stuff back.

 

DataFrame[Event_Class: string, Event_Subclass: string, Current_Time: timestamp, Text_Data: string, Start_Time: timestamp, End_Time: timestamp, Duration: bigint, Cpu_Time: bigint, Success: string, Integer_Data: bigint, Object_ID: string, Table_Name: string, Partition_Name: string, Start: double, End: double]

 

 

It's not until I restart the kernel and rerun the read to I get actual rows and columns back. 

dgwilson_0-1740078616436.png

 

What's going on? How can I ensure I get data back from my query without a kernel restart (or session stop/start)?

 

- David

1 ACCEPTED SOLUTION
dgwilson
Resolver III
Resolver III

I believe I have solved this.

 

I had the following in the code:

from IPython.display import display


This meant that when I called display(df) it gave me some output that was unexpected.
When I stopped and started the kernel and ran a separate code block to "do the same thing" the import was not rerun and I got different results.

Learning...

View solution in original post

1 REPLY 1
dgwilson
Resolver III
Resolver III

I believe I have solved this.

 

I had the following in the code:

from IPython.display import display


This meant that when I called display(df) it gave me some output that was unexpected.
When I stopped and started the kernel and ran a separate code block to "do the same thing" the import was not rerun and I got different results.

Learning...

Helpful resources

Announcements
July 2025 community update carousel

Fabric Community Update - July 2025

Find out what's new and trending in the Fabric community.

June FBC25 Carousel

Fabric Monthly Update - June 2025

Check out the June 2025 Fabric update to learn about new features.