Help with Authentication failed and not presented in the request when reading a CSV file

[dungtran12]

Hello,

   I  tried the Micrososft Fabric tutorial on saving and reading the CSV file to and from the Lakehouse.

https://learn.microsoft.com/en-us/fabric/data-science/tutorial-data-science-ingest-data since I used the code from the notebook.

It worked fine on writting the csv file to the Lakehouse. However, it did not work when I tried to read the same csv file from the Lakehouse. The Notebook compiler complained on something like authentication failed and not presented on the request.

{"error":{"code":"Unauthorized","message":"Authentication Failed with Bearer token is not present in the request"}}

I was in my workspace and I had my CVS file presented in the Lakehouse. I did not know why it complaint about authentication on the request.  Could you please help to figure it out  what was missing since I used the exact same codes from the tutorial?

##df = (
##    spark.read.option("header", True)
##    .option("inferSchema", True)
##    .csv("/lakehouse/default/Files/churn/raw/churn.csv") 
##    .cache()   
##)


df = spark.read.format ("csv").option("header","true").load ("/lakehouse/default/Files/churn/raw/churn.csv")

I was in my workspace and I had my CVS file presented in the Lakehouse so it should not complaint about authentication.

Lakehouse

[dungtran12]

Hello,

   I would like to update that it worked when I used the pandas read as  followed:

df = pd.read_csv("abfss://MinhTRanWorkspace@onelake.dfs.fabric.microsoft.com/MinhTRanLakehouse.Lakehouse/Files/churn/raw/churn.csv")
df.head()

 However, the Spark read still did not work. It still complaint about authentication failed and not presented in the request when reading a CSV file

{"error":{"code":"Unauthorized","message":"Authentication Failed with Bearer token is not present in the request"}}

Do you know what was missing with the Spark read?

Thanks,

Du

##df = (
##    spark.read.option("header", True)
##    .option("inferSchema", True)
##    .csv("/lakehouse/default/Files/churn/raw/churn.csv") 
##    .cache()   
##)


df = spark.read.format ("csv").option("header","true").load ("/lakehouse/default/Files/churn/raw/churn.csv")

[dungtran12 · Accepted solution]

Hello,

   It worked! I I used the spark read api to load data from csv file from the lakehouse.

df = spark.read.format("csv").option("header","true").load("Files/churn/raw/churn.csv")
display(df)

Thanks,

Du