Forum Discussion
Solved
How do you send mssparkutils.fs.ls() output to a dataframe
I can loop through the output using files = mssparkutils.fs.ls('Files/orders/')
for file in files:
print(file.name, file.isDir, file.isFile, file.path, file.size) But how do I se...
Hi PeteSpillane ,
You can do this with the following code
from notebookutils import mssparkutils # Initialise variables data = [] columns = ["File Name", "Is Dir", "Is File", "File Path", "File Size"] files = mssparkutils.fs.ls('Files/orders/') # Add rows to lists for file in files: data.append([file.name, file.isDir, file.isFile, file.path, file.size]) # Create a dataframe dataframe = spark.createDataFrame(data, columns) # Show data frame dataframe.show()Tested my side in Fabric notebook and all seemed to work okay.
Hope it helps,
Kris
kriscoupe
Solution Supplier
Solution SupplierHi PeteSpillane ,
You can do this with the following code
from notebookutils import mssparkutils
# Initialise variables
data = []
columns = ["File Name", "Is Dir", "Is File", "File Path", "File Size"]
files = mssparkutils.fs.ls('Files/orders/')
# Add rows to lists
for file in files:
data.append([file.name, file.isDir, file.isFile, file.path, file.size])
# Create a dataframe
dataframe = spark.createDataFrame(data, columns)
# Show data frame
dataframe.show()
Tested my side in Fabric notebook and all seemed to work okay.
Hope it helps,
Kris