Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Win a FREE 3 Day Ticket to FabCon Vienna. Apply now

Reply
PeteSpillane
Frequent Visitor

How do you send mssparkutils.fs.ls() output to a dataframe

I can loop through the output using 

 

 

files = mssparkutils.fs.ls('Files/orders/')
for file in files:
    print(file.name, file.isDir, file.isFile, file.path, file.size)

 

 

But how do I send the output to a dataframe instead?

1 ACCEPTED SOLUTION
kriscoupe
Solution Supplier
Solution Supplier

Hi @PeteSpillane ,

 

You can do this with the following code

 

from notebookutils import mssparkutils

# Initialise variables
data = []
columns = ["File Name", "Is Dir", "Is File", "File Path", "File Size"]
files = mssparkutils.fs.ls('Files/orders/')

# Add rows to lists
for file in files:
    data.append([file.name, file.isDir, file.isFile, file.path, file.size])

# Create a dataframe 
dataframe = spark.createDataFrame(data, columns) 
  
# Show data frame
dataframe.show()

 

Tested my side in Fabric notebook and all seemed to work okay.

 

Hope it helps,

Kris

View solution in original post

2 REPLIES 2
PeteSpillane
Frequent Visitor

Works perfectly.  Thanks Kris!

kriscoupe
Solution Supplier
Solution Supplier

Hi @PeteSpillane ,

 

You can do this with the following code

 

from notebookutils import mssparkutils

# Initialise variables
data = []
columns = ["File Name", "Is Dir", "Is File", "File Path", "File Size"]
files = mssparkutils.fs.ls('Files/orders/')

# Add rows to lists
for file in files:
    data.append([file.name, file.isDir, file.isFile, file.path, file.size])

# Create a dataframe 
dataframe = spark.createDataFrame(data, columns) 
  
# Show data frame
dataframe.show()

 

Tested my side in Fabric notebook and all seemed to work okay.

 

Hope it helps,

Kris

Helpful resources

Announcements
August Fabric Update Carousel

Fabric Monthly Update - August 2025

Check out the August 2025 Fabric update to learn about new features.

August 2025 community update carousel

Fabric Community Update - August 2025

Find out what's new and trending in the Fabric community.