Forum Discussion
Error while writing PySpark code to load the csv file data into a dataframe
- 1 year ago
Hi Anonymous
Thanks for using Microsoft Fabric Community.
The error you are facing is likely due to there is no Lakehouse attached to the Notebook.
From the images , in the Explorer pane under Resources there is no Lakehouse added.Please add the Lakehouse where the CSV file exists and retry.
Alternatively, you can include the ABFS path in the code to avoid attaching the Lakehouse.:
Here is the updated code:from pyspark.sql import SparkSession
spark = SparkSession.builder.appName("CSV Reader").getOrCreate()
# ABFS Path for the file
abfs_path = "Your ABFS path here"
df = spark.read.format("csv") \
.option("header", "true") \
.load(abfs_path)
display(df.limit(10)To obtain the ABFS path for the CSV file:
Navigate to Lakehouse -> Files -> data -> products.csv, then check the properties of the csv to copy the ABFS path.
Hope it works.
If this post helps please accept as solution to help others find it and a kudos would be appreciated.
Best Regards,
Vinay
Hi Anonymous ,
We just wanted to check in again regarding your issue. If you’ve found a solution, marking the reply as the solution and leaving a kudos would be greatly appreciated—it helps the community and others with similar questions.
If you’re still facing challenges or have further questions, please let us know—we’re here to help and would love to assist you in resolving this.
Looking forward to hearing back from you!
Best regards,
Vinay.