Forum Discussion

diablo9081's avatar
diablo9081
Frequent Visitor
6 months ago
Solved

Authentication in Fabric User Defined Function

I'm trying to use delta-rs write_deltalake() in a Fabric UDF that I'm converting from testing in a Fabric notebook. I've got the following code:

 

storage_options = {'bearer_token': notebookutils.credentials.getToken('storage'), 'use_fabric_endpoint': 'true'}

table_path = 'abfss://[workspace_id]@onelake.dfs.fabric.microsoft.com/[lakehouse_id]/Tables/dbo/[table_name]'

write_deltalake(table_path, data=pa.Table.from_pandas(df), mode='overwrite', storage_options=storage_options)

 

This works great in my notebook and writes the data in my pandas dataframe to the table in my lakehouse

 

I'm trying to put this code in a Fabric UDF, but I don't know how to authenticate properly. I've tried using credential = DefaultAzureCredential(), but it fails.

 

How can I properly authenticate in my Fabric UDF to do what those three lines above do so easily?

one lake
security
  • deborshi_nag's avatar
    deborshi_nag
    6 months ago

    Hello diablo9081

     

    Create a connection to your target Lakehouse inside your UDF item (Manage connections > Add data connection). Note the generated alias; you’ll have to put it in the decorator. Lakehouse connections provide read/write to Files.

     

    Add libraries to the UDF item (Toolbar → Library management):

    polars (for the DataFrame)

    Publish the library changes. Public PyPI packages are supported in UDFs.

     

    In a User data functions (UDF) item you don’t have NotebookUtils (the usual way to get a bearer token). You can use the following code that uses a SPN to authenticate -
     
    from azure.identity import ClientSecretCredential
from azure.storage.filedatalake import DataLakeServiceClient

tenant_id    = "<tenant_guid>"
client_id    = "<app_client_id>"
client_secret= "<client_secret>"

cred = ClientSecretCredential(tenant_id, client_id, client_secret)

svc = DataLakeServiceClient(
    account_url="https://onelake.dfs.fabric.microsoft.com",
    credential=cred
)

     

3 Replies

  • deborshi_nag's avatar
    deborshi_nag
    Super User
    6 months ago

    Hello diablo9081

     

    Create a connection to your target Lakehouse inside your UDF item (Manage connections > Add data connection). Note the generated alias; you’ll have to put it in the decorator. Lakehouse connections provide read/write to Files.

     

    Add libraries to the UDF item (Toolbar → Library management):

    polars (for the DataFrame)

    Publish the library changes. Public PyPI packages are supported in UDFs.

     

    In a User data functions (UDF) item you don’t have NotebookUtils (the usual way to get a bearer token). You can use the following code that uses a SPN to authenticate -
     
    from azure.identity import ClientSecretCredential
from azure.storage.filedatalake import DataLakeServiceClient

tenant_id    = "<tenant_guid>"
client_id    = "<app_client_id>"
client_secret= "<client_secret>"

cred = ClientSecretCredential(tenant_id, client_id, client_secret)

svc = DataLakeServiceClient(
    account_url="https://onelake.dfs.fabric.microsoft.com",
    credential=cred
)

     

  • v-prasare's avatar
    v-prasare
    Community Support
    5 months ago

    Hi diablo9081,

    We would like to confirm if our community members answer resolves your query or if you need further help. If you still have any questions or need more support, please feel free to let us know. We are happy to help you.


    deborshi_nag ,Thanks for your prompt response

     

     

    Thank you for your patience and look forward to hearing from you.
    Best Regards,
    Prashanth Are
    MS Fabric community support

  • v-prasare's avatar
    v-prasare
    Community Support
    5 months ago

    Hi @diablo9081,

    We would like to confirm if our community members answer resolves your query or if you need further help. If you still have any questions or need more support, please feel free to let us know. We are happy to help you.

     

     

     

    Thank you for your patience and look forward to hearing from you.
    Best Regards,
    Prashanth Are
    MS Fabric community support