Job Instance API returning inconsistent and incorrect results

[dafad_dew]

The List Job Instances API appears to be returning inconsistent and incorrect results.

I call the List Job Instances endpoint from my "Master" pipeline to get its job instances, and I get a set of records back which includes the current run.

URL for web activity is

https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/items/{master pipeline id}/jobs/instances

Issue 1.

The current run of "Master" has a status of "NotStarted".

It must have started - otherwise the Web activity would not have executed!!!

So there appears to be a delay between the current pipeline status and the status being returned by the API.  

Issue 2.

I copy the Web activity which calls the API into a pipeline called "Child".

The Web activities in Master and Child use the exact same URL - same workspace id and Master pipeline id. 

I add an Invoke Pipeline (Legacy) to Master and get it to run Child in addition to its own Web activity.

The Web activity in Master returns the current run, with a status of "NotStarted" - Issue 1 above.

The Web activity in Child does not include the current run, only completed runs.

On multiple attempts, I cannot get the Master run to be included in the API response called from Child.

Any clues as to why this would be? Is the response of the API dependent on the pipeline it gets called from?!

Hmmm.....

[v-prasare]

Hi @dafad_dew,

 As we haven’t heard back from you, we wanted to kindly follow up to check if the issue got resolved? or let us know if you need any further assistance here?

Thanks,

Prashanth Are

MS Fabric community support

[v-prasare]

Hi @dafad_dew , As we haven’t heard back from you, we wanted to kindly follow up to check if the issue got resolved? or let us know if you need any further assistance here?

Thanks,

Prashanth Are

MS Fabric community support

[v-prasare]

Hi @dafad_dew , As we haven’t heard back from you, we wanted to kindly follow up to check if the issue got resolved? or let us know if you need any further assistance here?

Thanks,

Prashanth Are

MS Fabric community support

[v-prasare · Accepted solution]

Hi @dafad_dew,

Thanks for reaching MS Fabric community support.

This is possibly a generic issue tied to distributed metadata synchronization in distributed systems (like Microsoft Fabric) where data updates (such as job status) don’t instantly reflect across all parts of the system. Instead, changes "eventually" reach a consistent state after a short delay.

Can you try below troubleshooting steps and let me know if this helps:

• Add a short delay before calling the API. Or use retry logic to check again after a few seconds if the status doesn't look correct
• Avoid relying on real-time visibility of job status from outside the pipeline that's running. due to eventual consistency you may not get accurate results. this relay on how job metadata is propagated within Fabric

Thanks,

Prashanth Are

MS Fabric community support

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query