Forum Discussion

ati_puri's avatar
ati_puri
Resolver III
4 months ago
Solved

Lakehouse Cold Start

Hi Team, There is a Production workspace with customer which is not Live yet, however, when the internal Business users are querying a specific report via intercal client API, the query gets timeout...
  • Lodha_Jaydeep's avatar
    4 months ago

    Hi ati_puri ,
    Thanks for reaching fabric community. will always happy to assist.

     

    You've correctly diagnosed the issue. This is the Fabric SQL Analytics Endpoint cold start problem a well-known platform behaviour. Let me give you the honest picture along with practical workarounds.

     

    Why This Happens

    The first execution of a query can be unexpectedly slower than subsequent runs this is a cold start caused by system initialization. Data is loaded from OneLake into memory because it's being accessed for the first time and isn't yet cached. Fabric automatically pauses nodes after a period of inactivity to reduce cost, and resuming them adds to query duration. Microsoft Learn

     

    The ~30 second delay you're seeing is exactly this resume + cache-load time.

     

    Workaround 1 Scheduled "Keep Warm" Pipeline (Most Practical)

    Create a lightweight Fabric Pipeline that runs a simple dummy query against the Lakehouse SQL Endpoint every 10–12 minutes (within the ~15 min cache window you mentioned).

    SELECT TOP 1 * FROM your_schema.your_table

     

    Schedule this as a recurring pipeline trigger. It keeps the endpoint warm so real user queries never hit a cold state. This is the most reliable self-service fix available today.

     

    Workaround 2 Add Retry Logic in Your API Layer

    Microsoft recommends adding retry logic with delay to handle transient cold start delays use exponential backoff to avoid retry storms. Microsoft Learn

     

    In your intercal client API, implement:

    • Retry up to 3 times with a 10–15 second wait between attempts
    • This handles the cold start transparently without the user ever seeing a timeout

    This is actually the architecturally cleaner fix compared to keep-warm, because it handles unexpected cold starts even if the warm-up pipeline misses a window.

     

    Workaround 3 Run a Warm-Up Query at App Startup

    If your client API has an initialisation/startup phase, fire a lightweight background query to the SQL endpoint when the app loads before any user triggers a real query. This warms the cache proactively at session start.

     

    Workaround 4 Pre-compute with Direct Lake Semantic Model

    If your reports are Power BI based, consider switching to a Direct Lake Semantic Model on top of the Lakehouse. Direct Lake has its own warm-up mechanism and is significantly more optimised for report query performance compared to hitting the SQL endpoint directly via API.

     

    There is no native "keep warm" setting in Fabric as of now Microsoft doesn't expose a configuration to prevent node pausing on the Lakehouse SQL Endpoint. The keep-warm pipeline is a community workaround, not an official feature. Worth submitting this as a feature request on the Fabric Ideas Forum it's a real gap.

     

    Recommended Approach for Go-Live

    Priority

    Action

    Must have

    Retry logic with backoff in your API layer

    Recommended

    Scheduled keep-warm pipeline every 10 min

    Nice to have

    Warm-up query on app/session startup

     

    Combine Workarounds 1 + 2 for the most robust solution before go-live. Hope this helps!

     

    Please consider marking this as the Accepted Solution to help other community members find this fix more easily. If this helped you, Kudos are always appreciated!