Write_delta in Fabric Python Notebook Error

[Ka13]

Hi ,

I had question in Fabric Python Notebook while using the write_deltalake was getting Error.

I want to write the Data from Pandas Dataframe in the Lakehouse Table - cust_test.

Can you please suggest.

Below is the Fabric Notebook code.



OSError: Generic MicrosoftAzure error: URL did not match any known pattern for scheme: abfss://<workspace-name>@onelake.dfs.fabric.microsoft.com/test.Lakehouse/Tables/cust_test



Fabric Notebook -

pip install deltalake


import pandas as pd
from deltalake import write_deltalake
from datetime import datetime, timezone
from notebookutils import notebook

data = {
    "CustomerID": [1, 2, 3],
    "Name": ["Alice", "Bob", "Charlie"],
    "Age" : [33,27,22]
}

df = pd.DataFrame(data)
print(df.info())



storage_options = {
    "bearer_token" : notebookutils.credentials.getToken('storage'),
    "use_fabric_endpoint": "true"
}



abfss_path = "abfss://<workspace-name>@onelake.dfs.fabric.microsoft.com/test.Lakehouse/Tables/cust_test"




write_deltalake(
    abfss_path,
    df,
    mode="append",
    storage_options = storage_options
)  

[deborshi_nag]

Hello @Ka13 

If you don't need to write in Delta format you can simply write your pandas dataset using the "File API Path". Add the lakehouse as a Data item in your Python notebook first. In the /Files section of the lakehouse, click the three dots on the folder you want to write to, and from the context menu copy the File API Path option. You can simply use that with Pandas to_csv method to write the dataset into the folder. 

df.to_csv('/lakehouse/default/Files/pandas_data/customers.csv', index=False, encoding="utf-8")

For writing in delta file format it's best to use a Spark notebook and the abfs file path, because Delta is a Spark-native storage format!