Forum Discussion

Yazdan's avatar
Yazdan
Advocate II
6 months ago

How to programmatically access “Submitted by” (pipeline runner) at pipeline or notebook runtime

Hi Fabric team,

 

In the Fabric Monitor view, pipeline runs clearly show the “Submitted by” column (see bellow), which correctly reflects the user who manually triggered the pipeline (e.g. Yazdan Aghajanian).

 

However, I have not been able to access this information programmatically from either:

  • Pipeline runtime expressions

  • Notebooks executed by the pipeline

  • Fabric REST APIs

What I’ve tried (and confirmed):

  • Pipeline runtime / expressions
    There is no system variable or expression (e.g. @pipeline().submittedBy) that exposes the runner identity.

  • Notebook runtime context

     
    mssparkutils.runtime.context["userName"]

    This returns the execution identity, not the pipeline submitter, when the notebook is invoked by a pipeline.

  • Fabric REST APIs

    • GET /v1/workspaces/{workspaceId}/items/{pipelineItemId}/jobs/instances

    • GET /v1/workspaces/{workspaceId}/items/{pipelineItemId}/jobs/instances/{instanceId}

    • Activity endpoints under the instance

    These APIs only return execution metadata (status, invokeType, timestamps, rootActivityId, etc.).
    There is no field such as submittedBy, createdBy, or user principal information.

  • Azure Monitor / Graph APIs
    Not usable in our tenant (ARM/Graph token minting is blocked), and even then, this feels indirect given the data already exists in Fabric Monitor.

The core question

How can we access the “Submitted by” value (the user who triggered the pipeline) at pipeline or notebook runtime?

Specifically:

  • Is there a supported API or pipeline variable that exposes this?

  • Is the Monitor UI using an internal control-plane API that is not yet publicly exposed?

  • Is passing the submitter explicitly as a pipeline parameter the only supported approach?

This information is important for auditing and operational logging, and currently the Monitor UI has it, but user code cannot access it.

Any official guidance or roadmap insight would be greatly appreciated.

 

Thanks in advance.

data engineering
Environment
User Data Functions

13 Replies

  • deborshi_nag's avatar
    deborshi_nag
    Super User
    6 months ago

    Hello Yazdan 

     

    As a workaround you can enable Workspace monitoring on the workspace and query the KQL database. 

     

    In the workspace > Settings > Monitoring > turn on Log workspace activity (Fabric creates the monitoring Eventhouse + KQL DB).
    In the KQL DB, run a query like

    ItemJobEventLogs
| where ItemKind == "Pipeline"
| project Timestamp, WorkspaceName, ItemName, JobStatus,
          ExecutingPrincipalType, ExecutingPrincipalId, DurationMs
| order by Timestamp desc

     

    • Yazdan's avatar
      Yazdan
      Advocate II
      6 months ago

      Hi Deborshi,

      Thanks for the reply. However, in the result of this query I can’t see the user who triggered the pipeline. I need that information to be available in the pipeline environment at runtime.

      • deborshi_nag's avatar
        deborshi_nag
        Super User
        6 months ago

        Hi Yazdan Unfortunately, you won't be able to capture the Submitted By of a pipeline at runtime. 

  • deborshi_nag's avatar
    deborshi_nag
    Super User
    6 months ago

    Hello Yazdan 

     

    Unfortunately the Fabric Job Scheduler REST APIs don’t currently return “Submitted By” (the user who started the run) for DataPipeline job instances. The payloads you’re already using—List Item Job Instances and Get Item Job Instance—include fields like status, invokeType (Manual / Schedule / API), time stamps, and failureReason, but no user identity.

     

    This has also been answered in the Community forum in the past. 

     

    Solved: Is it possible to get username of the user who tri... - Microsoft Fabric Community

     

    • Yazdan's avatar
      Yazdan
      Advocate II
      6 months ago

      Hi Tayloramy,

      Thanks for the reply,
      How can we access the pipeline activity logs in the pipeline environment at runtime?
      Do you have any sample code for this?
      Thanks,

      • tayloramy's avatar
        tayloramy
        Super User
        6 months ago

        Hi Yazdan, 

         

        Once you enable Workspace Monitoring, you should be able to access the logs in the workspace monitoring eventhouse that is created. 

         

        I believe these logs include who executed the pipeline. 

         

    • v-dineshya's avatar
      v-dineshya
      Community Support
      6 months ago

      Hi Yazdan ,

      could you please try the proposed solution shared by  tayloramy ? Let us know if you’re still facing the same issue we’ll be happy to assist you further.

       

      Regards,

      Dinesh

      • Yazdan's avatar
        Yazdan
        Advocate II
        6 months ago

        Hi v-dineshya ,

        Thanks for reaching out. I’m not fully sure about the solution yet and was just testing at this stage. If it turns out to be the right approach, I’ll definitely share and mark it here as the solution.

        Appreciate your follow-up.

        Regards,
        Yazdan