Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Score big with last-minute savings on the final tickets to FabCon Vienna. Secure your discount

Reply
PeteSpillane
Frequent Visitor

How do you send mssparkutils.fs.ls() output to a dataframe

I can loop through the output using 

 

 

files = mssparkutils.fs.ls('Files/orders/')
for file in files:
    print(file.name, file.isDir, file.isFile, file.path, file.size)

 

 

But how do I send the output to a dataframe instead?

1 ACCEPTED SOLUTION
kriscoupe
Solution Supplier
Solution Supplier

Hi @PeteSpillane ,

 

You can do this with the following code

 

from notebookutils import mssparkutils

# Initialise variables
data = []
columns = ["File Name", "Is Dir", "Is File", "File Path", "File Size"]
files = mssparkutils.fs.ls('Files/orders/')

# Add rows to lists
for file in files:
    data.append([file.name, file.isDir, file.isFile, file.path, file.size])

# Create a dataframe 
dataframe = spark.createDataFrame(data, columns) 
  
# Show data frame
dataframe.show()

 

Tested my side in Fabric notebook and all seemed to work okay.

 

Hope it helps,

Kris

View solution in original post

2 REPLIES 2
PeteSpillane
Frequent Visitor

Works perfectly.  Thanks Kris!

kriscoupe
Solution Supplier
Solution Supplier

Hi @PeteSpillane ,

 

You can do this with the following code

 

from notebookutils import mssparkutils

# Initialise variables
data = []
columns = ["File Name", "Is Dir", "Is File", "File Path", "File Size"]
files = mssparkutils.fs.ls('Files/orders/')

# Add rows to lists
for file in files:
    data.append([file.name, file.isDir, file.isFile, file.path, file.size])

# Create a dataframe 
dataframe = spark.createDataFrame(data, columns) 
  
# Show data frame
dataframe.show()

 

Tested my side in Fabric notebook and all seemed to work okay.

 

Hope it helps,

Kris

Helpful resources

Announcements
August 2025 community update carousel

Fabric Community Update - August 2025

Find out what's new and trending in the Fabric community.