Forum Discussion

Ka13's avatar
Ka13
Icon for Helper II rankHelper II
7 months ago
Solved

Write_delta in Fabric Python Notebook Error

Hi ,   I had question in Fabric Python Notebook while using the write_deltalake was getting Error.   I want to write the Data from Pandas Dataframe in the Lakehouse Table - cust_test. Can yo...
data engineering
lakehouse
notebook
  • deborshi_nag's avatar
    deborshi_nag
    7 months ago

    Hello Ka13 I doubt you'd be able to import mssparkutils as it is a Spark utility and you're using a Python notebook. 

     

    Here's a code you can use in Python notebook -

     

    import sempy.fabric as fabric

def get_workspace_and_lakehouse_id(target_name="my_lakehouse"):
    # Workspace ID
    workspace_id = fabric.get_notebook_workspace_id()

    # List lakehouse items (your exact signature)
    items = fabric.list_items(type="Lakehouse", workspace=workspace_id)

    lakehouse_id = None

    # Loop through rows (your structure: it[0] = id, it[1] = name)
    for it in items.values:
        if it[1] == target_name:
            lakehouse_id = it[0]
            break

    return workspace_id, lakehouse_id

print(get_workspace_and_lakehouse_id())

     

deborshi_nag's avatar
deborshi_nag
Icon for Super User rankSuper User
7 months ago

Hello Ka13 

 

If you don't need to write in Delta format you can simply write your pandas dataset using the "File API Path". Add the lakehouse as a Data item in your Python notebook first. In the /Files section of the lakehouse, click the three dots on the folder you want to write to, and from the context menu copy the File API Path option. You can simply use that with Pandas to_csv method to write the dataset into the folder. 

 

df.to_csv('/lakehouse/default/Files/pandas_data/customers.csv', index=False, encoding="utf-8")

 

For writing in delta file format it's best to use a Spark notebook and the abfs file path, because Delta is a Spark-native storage format!