Forum Discussion

zhaya's avatar
zhaya
Icon for Microsoft Employee rankMicrosoft Employee
6 months ago
Solved

Looking for feedback: Fabric Job Public API throttling – what are your real scenarios?

Hi everyone I’m part of the Microsoft Fabric team, working on the Fabric Job Public APIs. We’ve heard feedback from several customers (FabCon, interviews, direct conversations) that the current thr...
  • tayloramy's avatar
    tayloramy
    6 months ago

    Hi zhaya, 

     

    The perfectionist in me wants to poll the API every few seconds so there is the least delay possible in detecting that one job finished successfully and kicking off the next job, though that might not be realistic unless there was a websocket api available. 

     

    I switched my stuff over to poll every 5 minutes I think and that resolved the API errors, though waiting 5 minutes between jobs can add up in very large enviornments. 

     

    You said that the current limit is applied at the item level? Has that always been the case? 
    If that is true, then I should be able to poll every running job every second without hitting limits, as that would only be 60 requests a minute. That is not what I experienced. 

    Some background for what I was doing: 
    I was working in a rather large environment, we had 3 F64 and 2 F128 capacities. 
    I was trying to build a system to robustly orchestrate jobs with complex dependency chains (so using a Pipeline to orchestrate wasn't enough) 

    I'm no longer at the company I was at when I was doing this, so I don't have any of the code anymore.