Forum Discussion

Hansie151's avatar
Hansie151
New Member
18 hours ago

Fabric UDFs Deployment ISSUEs

When using python packages the UFDs work in testing mode, but when published and running in run mode I get the following error:

Error:

{   "functionName": "hello_fabric",   "invocationId": "00000000-0000-0000-0000-000000000000",   "status": "Failed",   "errors": [     {       "errorCode": "WorkloadException",       "subErrorCode": "NotFound",       "message": "User data function: 'hello_fabric' invocation failed."     }   ] }

I have pinpointed that this only happens when adding the bigquery auth package and also only in prod/run mode. I have no idea how to fix it as it was working for the past month but after small change to my functions the publishing always show this error.

Example function:

import datetime import fabric.functions as fn import logging import json from google.cloud import bigquery from google.oauth2.credentials import Credentials  udf = fn.UserDataFunctions()   def get_bigquery_client(var_lib: fn.FabricVariablesClient, auth_type: str = "SERVICE") -> bigquery.Client:     variables = var_lib.getVariables()     auth_type_upper = auth_type.upper()      if auth_type_upper == "PERSONAL":         json_string = variables.get("GCP_PERSONAL_API_CREDS_JSON") or ""         if not json_string:             raise ValueError("auth_type is 'PERSONAL' but GCP_PERSONAL_API_CREDS_JSON is missing or empty.")          credentials_info = json.loads(json_string)         project_id = credentials_info.get("quota_project_id")         user_creds = Credentials.from_authorized_user_info(credentials_info)         return bigquery.Client(credentials=user_creds, project=project_id)      if auth_type_upper == "SERVICE":         json_string = variables.get("GCP_API_CREDS_JSON") or ""         if not json_string:             raise ValueError("auth_type is 'SERVICE' but GCP_API_CREDS_JSON is missing or empty.")          credentials_info = json.loads(json_string)         return bigquery.Client.from_service_account_info(credentials_info)      raise ValueError(f"Invalid auth_type '{auth_type}'. Must be 'SERVICE' or 'PERSONAL'.")  @udf.connection(argName="varLib", alias="apivariables") @udf.function() def hello_fabric(varLib: fn.FabricVariablesClient,name: str) -> str:     client = get_bigquery_client(varLib, "PERSONAL")     logging.info('Python UDF trigger function processed a request.')      return f"{varLib} {client} Welcome to Fabric Functions, {name}, at {datetime.datetime.now()}!"

Any help would be appreciated!

 

User Data Functions

2 Replies

  • odtJitendra's avatar
    odtJitendra
    Advocate I
    17 hours ago

    Hi Hansie151​,

    Yes — the pattern you describe strongly points to a Fabric UDF dependency/runtime mismatch, rather than a problem with your BigQuery credentials or the get_bigquery_client() logic.

    The key clue is: Works in Test/Develop mode, fails only after Publish → Run only mode, and adding google-cloud-bigquery triggers it.

    Please check this.

     

  • maxravi's avatar
    maxravi
    Regular Visitor
    16 hours ago

    This looks like a Fabric User Data Functions (UDF) deployment/runtime issue rather than an issue with your Python code itself.

     

    Since the function works in Test mode but fails in Run/Published mode, and you have isolated the problem to the "google-cloud-bigquery" / "google.oauth2" packages, I would check the production dependency resolution first.

     

    A few things I would try:

     

    1. Pin the package versions instead of relying on the latest versions. For example, explicitly define compatible versions of:

       

       - "google-cloud-bigquery"

       - "google-auth"

       - "google-api-core"

     

    2. Check the published environment's dependency list and confirm that the Google packages are actually being included after publishing. Test mode can sometimes have a different dependency/runtime context than the deployed function.

     

    3. Try importing the packages separately:

       

       import google.auth

    import google.oauth2.credentials

    import google.cloud.bigquery

       

       This can help identify whether the failure occurs during module initialization.

     

    4. Since the error is only:

       "WorkloadException / NotFound"

       

       and does not expose the underlying Python exception, check the Fabric monitoring/logging output for the published invocation. The actual import/dependency error may be hidden behind the generic workload error.

     

    5. As a diagnostic, temporarily remove the BigQuery imports and the "get_bigquery_client()" call. If the published function runs successfully, that further confirms the dependency/runtime issue.

     

    One other thing I'd test is changing:

     

    from google.oauth2.credentials import Credentials

     

    to a lazy import inside the "PERSONAL" branch. That prevents the Google OAuth package from being loaded when the function is initialized:

     

    if auth_type_upper == "PERSONAL":

        from google.oauth2.credentials import Credentials

        ...

     

    Given that this worked previously and started failing after a relatively small change, I'd also compare the package versions and Fabric runtime/deployment configuration between the last working publication and the current one.

     

    If all of this checks out, I'd raise it with Microsoft as a Fabric Functions deployment regression, providing the function definition, dependency versions, and the fact that Test mode succeeds while Published/Run mode returns "WorkloadException: NotFound".

    Please let me know if any of the above solution works for you .