Forum Discussion
Error while writing PySpark code to load the csv file data into a dataframe
- 1 year ago
Hi Anonymous
Thanks for using Microsoft Fabric Community.
The error you are facing is likely due to there is no Lakehouse attached to the Notebook.
From the images , in the Explorer pane under Resources there is no Lakehouse added.Please add the Lakehouse where the CSV file exists and retry.
Alternatively, you can include the ABFS path in the code to avoid attaching the Lakehouse.:
Here is the updated code:from pyspark.sql import SparkSession
spark = SparkSession.builder.appName("CSV Reader").getOrCreate()
# ABFS Path for the file
abfs_path = "Your ABFS path here"
df = spark.read.format("csv") \
.option("header", "true") \
.load(abfs_path)
display(df.limit(10)To obtain the ABFS path for the CSV file:
Navigate to Lakehouse -> Files -> data -> products.csv, then check the properties of the csv to copy the ABFS path.
Hope it works.
If this post helps please accept as solution to help others find it and a kudos would be appreciated.
Best Regards,
Vinay
Hi Anonymous ,
We just wanted to follow up one last time to check if the issue has been resolved on your end. If you’re still encountering issues or need any further assistance, please don’t hesitate to reach out—we’re always happy to help.
Best regards,
Vinay.