Forum Discussion

smeetsh's avatar
smeetsh
Icon for Continued Contributor rankContinued Contributor
1 month ago
Solved

SQL Endpoint lakehouse: Intermittent error in the Refresh SQL analytics endpoint activity

Hi All, since Monday we are having intermittent issues with the Refresh SQL analytics endpoint in our pipelines in fabric.  We run multiple pipelines on an F8 sku, which Monday was about 60% used an...
data pipeline
lakehouse
  • smeetsh's avatar
    smeetsh
    26 days ago

    Thank you for all the responses, going down the full api/notebook route is too much of an overhead for us at the moment. It would require us to set up a service principal,  api keys etc, which we (don't) have (yet) (very small team :-) )

    We do know from experience that a 5-minute wait in our pipeline, though ugly, solved the issue. Endpoints would have synched by then. 

    (The only time that did not work flawless was on 1 pipeline that ran at 130 am in the morning, when the lakehouse was well deep asleep. I solved that by running a dummy script against the endpoint...select top (1) from etc etc.

    With that in mind I would think that (in our situation) "on completion" would be safe?

    Cheers

    Hans

v-sathmakuri's avatar
v-sathmakuri
Icon for Community Support rankCommunity Support
1 month ago

Hi smeetsh​ ,

Thank you for reaching out to fabric community.

  • This is most likely a transient Fabric API timeout not an F8 capacity issue.
  • Add a Retry policy with a delay to the Refresh SQL analytics endpoint activity.
  • Ensure the activity runs after all Lakehouse write / ingestion operations are completed to avoid concurrent update / locking issues.
  • Microsoft's current documentation specifically states that the activity can intermittently fail when other processes are actively updating the underlying Lakehouse, because the SQL analytics endpoint needs to acquire internal locks. However, the exact 408 Request Timeout / A task was canceled error in your example is not explicitly documented as the known issue.

Refresh SQL analytics endpoint activity - Microsoft Fabric | Microsoft Learn

Thanks!!