Forum Discussion

michael_muell's avatar
michael_muell
New Member
1 year ago
Solved

UDF Connection to Lakehouse does not work

Hi all,    I'm trying to get a UDF to work that creates a file in the lakehouse. I saw that there is a sample that almost achieves what I want. Unfortunately it does not work when I connect my lake...
  • v-venuppu's avatar
    1 year ago

    Hi michael_muell ,

    Please use the below code:

    import pandas as pd

    import datetime

    import fabric.functions as fn

    import logging

     

    udf = fn.UserDataFunctions()

     

    @udf.connection(argName="myLakehouse", alias="LH123")

    @udf.function()

    def write_csv_file_in_lakehouse(myLakehouse: fn.FabricLakehouseClient, employees: list) -> str:

        """

        Writes employee data to Lakehouse Files as a CSV.

        """

       

        logging.info("Starting CSV file write to Lakehouse")

     

        # Create timestamped filename

        csvFileName = "Employees_" + str(round(datetime.datetime.now().timestamp())) + ".csv"

       

        # Create DataFrame and CSV string

        df = pd.DataFrame(employees, columns=["ID", "EmpName", "DepID"])

        csv_string = df.to_csv(index=False)

        csv_bytes = csv_string.encode("utf-8")  # Convert string to bytes

     

        # Connect to Lakehouse Files and upload

        connection = myLakehouse.connectToFiles()

        file_client = connection.get_file_client(csvFileName)

        file_client.upload_data(csv_bytes, overwrite=True)

     

        # Close connections

        file_client.close()

        connection.close()

     

        return f"File '{csvFileName}' was uploaded successfully."

     

    Add Pandas library as shown below:

     

    To test this I have created pipeline and it worked for me:

     

    File got created in Lakehouse as shown below:

     

     

    Thank you.