Fabric User Data Function throws error while attempting call from data pipeline

[Mahalakshmi_15]

Requirement

Using REST API, we need to connect to Fabric Workspace to get data from delta table.

Solution Approach

The user data function in Fabric supports REST API to query data from delta table and it is in preview feature.

Fabric Tenant region is UK South

Implementation Steps

1. Registered the app in Azure portal. Given permission to access the storage, power BI and User data functions
2. Added the app id in Fabric workspace
3. Enabled the User data function in admin portal.
4. We created the user data function, published it and function is executing fine.
5. We enabled the public access property in user data function and republished.
6. Using Public URL, I tried to make the REST AP call using Python code - received 404 error message
7. As we got error message in previous step, we tried to call the user data function from fabric data pipeline, we received the error message below.

Kindly help me resolve this error.

[Mahalakshmi_15]

Hi Team,

Apologies for delayed responsed. 
In Fabric data pipeline, as you mentioned within the same tenant I'm able to execute it. 
Only through REST API call, I could not call the user data function(even after changing the token as mentioned). I beleive soon the Microsoft will fix this existing issue.

[v-pagayam-msft]

Hi @Mahalakshmi_15 ,
Has the issue been resolved on your end it? If so, please share your solution and mark it as "Accept as Solution." This will assist others in the community who are dealing with similar problems and help them find a solution more quickly.
Thank you.

[Mahalakshmi_15]

Adding a code block for this issue..

User Data function

import datetime

import fabric.functions as fn

import logging

udf = fn.UserDataFunctions()

#@udf.function()

# Replace the alias "<My Lakehouse alias>" with your connection alias.

@udf.connection(argName="myLakehouse", alias="lhreporting")

@udf.function()

def query_data_from_tables(myLakehouse: fn.FabricLakehouseClient) -> list:

    # Connect to the Lakehouse SQL Endpoint

    connection = myLakehouse.connectToSql()

    # Use connection to execute a query

    cursor = connection.cursor()

    cursor.execute(f"SELECT * FROM MD_FX_RATES;")

    rows = [x for x in cursor]

    columnNames = [x[0] for x in cursor.description]

    # Turn the rows into a json object

    values = []

    for row in rows:

        item = {}

        for prop, val in zip(columnNames, row):

            if isinstance(val, (datetime.date, datetime.datetime)):

                val = val.isoformat()

            item[prop] = val

        values.append(item)

    # Close the connection

    cursor.close()

    connection.close()

    return values

User Data function output

Fabric Data pipeline

Open the Fabric data pipeline à Add the function activity and configure the User Data function settings.

Fabric Data pipeline output

REST API Call Code

from azure.identity import InteractiveBrowserCredential

import requests

import json

from azure.identity import ClientSecretCredential

import pandas as pd   

import io

from pyspark.sql import SparkSession

# Acquire a token

# DO NOT USE IN PRODUCTION.

# Below code to acquire token is to test the GraphQL endpoint and is for the purpose of development only.

# For production, always register an application in a Microsoft Entra ID tenant and use the appropriate client_id and scopes.

# https://learn.microsoft.com/fabric/data-engineering/connect-apps-api-graphql#create-a-microsoft-entr...

tenant_id = '8bcc'

client_id = '9ee6'

client_secret = '1234'

scope = 'https://analysis.windows.net/powerbi/api/.default'

FUNCTION_URL = 'https://api.fabric.microsoft.com/v1/workspaces/43dd/userDataFunctions/5b8f/functions/query_data_from...'

# Authenticate and get the token

credential = ClientSecretCredential(tenant_id, client_id, client_secret)

token = credential.get_token(scope).token

headers = {

    'Authorization': f'Bearer {token}',

    'Content-Type': 'application/json'

}

try:  

    response = requests.get(FUNCTION_URL, headers=headers)

    response.raise_for_status()

    print(json.dumps(response.json()))

except Exception as e:

    print({"error": str(e)}, 500)

    print(response.headers)

REST API call output

[Mahalakshmi_15]

For my post, adding a code block for this issue for more analysis..

User Data function

import datetime

import fabric.functions as fn

import logging

udf = fn.UserDataFunctions()

#@udf.function()

# Replace the alias "<My Lakehouse alias>" with your connection alias.

@udf.connection(argName="myLakehouse", alias="lhreporting")

@udf.function()

def query_data_from_tables(myLakehouse: fn.FabricLakehouseClient) -> list:

    # Connect to the Lakehouse SQL Endpoint

    connection = myLakehouse.connectToSql()

    # Use connection to execute a query

    cursor = connection.cursor()

    cursor.execute(f"SELECT * FROM MD_FX_RATES;")

    rows = [x for x in cursor]

    columnNames = [x[0] for x in cursor.description]

    # Turn the rows into a json object

    values = []

    for row in rows:

        item = {}

        for prop, val in zip(columnNames, row):

            if isinstance(val, (datetime.date, datetime.datetime)):

                val = val.isoformat()

            item[prop] = val

        values.append(item)

    # Close the connection

    cursor.close()

    connection.close()

    return values

User Data function output

Fabric Data pipeline

Open the Fabric data pipeline à Add the function activity and configure the User Data function settings.

Fabric Data pipeline output

REST API Call Code

from azure.identity import InteractiveBrowserCredential

import requests

import json

from azure.identity import ClientSecretCredential

import pandas as pd   

import io

from pyspark.sql import SparkSession

# Acquire a token

# DO NOT USE IN PRODUCTION.

# Below code to acquire token is to test the GraphQL endpoint and is for the purpose of development only.

# For production, always register an application in a Microsoft Entra ID tenant and use the appropriate client_id and scopes.

# https://learn.microsoft.com/fabric/data-engineering/connect-apps-api-graphql#create-a-microsoft-entr...

tenant_id = '8bcc'

client_id = '9ee6'

client_secret = '1234'

scope = 'https://analysis.windows.net/powerbi/api/.default'

FUNCTION_URL = 'https://api.fabric.microsoft.com/v1/workspaces/43dd/userDataFunctions/5b8f/functions/query_data_from...'

# Authenticate and get the token

credential = ClientSecretCredential(tenant_id, client_id, client_secret)

token = credential.get_token(scope).token

headers = {

    'Authorization': f'Bearer {token}',

    'Content-Type': 'application/json'

}

try:  

    response = requests.get(FUNCTION_URL, headers=headers)

    response.raise_for_status()

    print(json.dumps(response.json()))

except Exception as e:

    print({"error": str(e)}, 500)

    print(response.headers)

REST API call output

[v-pagayam-msft]

Hi @Mahalakshmi_15 ,
Thank you for reachong ou to us!

Thanks againfor sharing the detailed steps and setup.

The error you are seeing when calling the User Data Function via REST API and "PowerBIEntityNotFound" error in the Fabric Data Pipeline likely stem from a known issue in Microsoft Fabric when working across different tenants. This issue occurs when a function, Dataflow Gen2, or pipeline tries to connect to a Lakehouse located in a different tenant.Microsoft team is actively working on to fix this issue.

Based on your setup, it's possible that the Lakehouse and the User Data Function or registered Azure app are in different tenants or regions.Please verify that the Lakehouse, User Data Function, and REST API call are all within the same tenant and region.

Also, as a work-arund, update the token scope in your REST API code.Double check that the function URL you are using is correct format , and that the app has all required permissions granted in Entra ID and Fabric.

You may find this scenario listed under the known issues for Microsoft Fabric here:
https://learn.microsoft.com/en-us/fabric/known-issues/fabric-known-issues 

Hope this resolve your query.If so,consider accepting tit as solution.

Regards,
Pallavi.

[Mahalakshmi_15]

Thank you for response. 

Could you please tell me the token scope that I need to use for REST API call. So i can try with updated scope value

[v-pagayam-msft · Accepted solution]

Hi @Mahalakshmi_15 ,
Thank you for the follow-up ! To make the REST API call to your User Data Function work, you may want to use like this token scope:
https://api.fabric.microsoft.com/.default  
This tells the system you’re trying to access Fabric-specific resources. Just make sure your app in Entra ID has the right API permissions for Microsoft Fabric, and that consent has been granted (either by you or an admin, depending on your setup). Once you update the scope and retry the call, it might bring you a step closer.

As I mentioed earlier,So even with the correct scope, you might still see errors . You can find more details in the known issues.

I hope this helps you to resolve your query.If so,consider acepting it as solution.

Regards,
Pallavi.