AnalysisException: Path does not exist:

[naveen73]

Hi,

I am doing this excercise:

https://microsoftlearning.github.io/mslearn-fabric/Instructions/Labs/04-ingest-pipeline.html

When I create a notebook, the following code gives an error message:

from pyspark.sql.functions import *

# Read the new sales data
df = spark.read.format("csv").option("header","true").load("Files/new_data/*.csv")

## Add month and year columns
df = df.withColumn("Year", year(col("OrderDate"))).withColumn("Month", month(col("OrderDate")))

# Derive FirstName and LastName columns
df = df.withColumn("FirstName", split(col("CustomerName"), " ").getItem(0)).withColumn("LastName", split(col("CustomerName"), " ").getItem(1))

# Filter and reorder columns
df = df["SalesOrderNumber", "SalesOrderLineNumber", "OrderDate", "Year", "Month", "FirstName", "LastName", "EmailAddress", "Item", "Quantity", "UnitPrice", "TaxAmount"]

# Load the data into a table
df.write.format("delta").mode("append").saveAsTable(table_name)

AnalysisException: Path does not exist: abfss://84d5de85-0a8e-490b-8585-bd3cb3542a51@onelake.dfs.fabric.microsoft.com/0fc55ae4-9a3c-40e3-9ae6-0a3155cc452e/Files/new_data/*.csv

Can you help me with this, please?

Thanks,

Naveen

[Syndicate_Admin]

This warning is for the output, i.e. you did not specify the destination of the data, in this case the Lakehouse you already have.