Forum Discussion
Fabric/Copilot late response
Copilot/Data Agent queries are consistently taking ~29s to return a response.
Data source is a single Gold table with ~4,000 records — tested against both the lakehouse table directly and a Direct Lake semantic model built on it, same latency either way.
Also tried Preview and Standard modes in Test Copilot — no meaningful difference between them.
Given the small data volume, this points to the NL→SQL orchestration pipeline (schema retrieval → SQL generation → execution → NL response) as the bottleneck, not the data source, table size, or capacity tier.
Is ~29s the expected baseline today, or is there a way to reduce the round-trips? Keen to hear if anyone's found a workaround.
Hi,
This is true and it is currently hard to bench mark the seconds as most of the leading LLM model provider does not provide the specific latency bench mark table for reference. Having said that below are some information that these Model providers highlights :
OpenAI states that latency is affected by:
- model size
- output tokens
- number of requests
- orchestration choices
Anthropic states that latency is affected by:
- model selection
- prompt size
- output size
- infrastructure
Also , Microsoft's own LLM evaluation guidance states that RAG/Copilot systems often:
"go through multiple iterations to retrieve relevant pieces of information until it is satisfied that it has enough information, and then have another LLM call to synthesize the response."
Looking all these and considering (on assumptions)
For a simple chat model call:
Metric Typical Range TTFT 0.5–3 sec Short answer 1–10 sec Complex reasoning 5–20 sec For an agentic workflow (Copilot, Data Agent, RAG):
Step Latency Schema retrieval 1–5 sec Planning 1–5 sec SQL generation 1–5 sec Execution <1 sec on 4K rows Answer synthesis 2–10 sec Total: 10–30+ seconds is not unusual for multi-step agents even when the source table is tiny
Hope this gives a new prospective of looking into the evolution till we have the complete clearity
7 Replies
- trivedisunita
Continued Contributor
Hi vishal307088 ,
Based on your testing, the data itself doesn’t seem to be the main issue. With only ~4,000 rows and almost the same response time from both the Lakehouse table and Direct Lake semantic model, the delay seems to be happening somewhere in the overall Data Agent processing rather than in the query execution.
As far as I know, Microsoft hasn’t published any specific expected response time or SLA for Data Agent queries, so I wouldnot consider ~29 seconds a confirmed baseline.
One think you could try is asking a very simple question (like- How many records are there?), multiple times and compare the results with a more complex analytical prompt. If both consistently return in roughly the same time ~29s range, it would be a indication that the delay isn’t really related to the amount of data or query complexity.I am not aware of any documented setting, that can significantly reduce the Data Agent orchestration steps.Based on what you've found so far, it may be worth looking more closely at the agent/runtime side before focusing on table or model optimization.Hope this helps! 
v-kathullacCommunity Support
Thankyou ShivekMaharaj , trivedisunita ShahRukhSameer for Addressing the issue.
Hi vishal307088 ,
As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?
Regards,
Chaithanya
- v-kathullac
Hi vishal307088 ,
As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?
Regards,
Chaithanya
- sandeephijam
Helper II
Hi,
This is true and it is currently hard to bench mark the seconds as most of the leading LLM model provider does not provide the specific latency bench mark table for reference. Having said that below are some information that these Model providers highlights :
OpenAI states that latency is affected by:
- model size
- output tokens
- number of requests
- orchestration choices
Anthropic states that latency is affected by:
- model selection
- prompt size
- output size
- infrastructure
Also , Microsoft's own LLM evaluation guidance states that RAG/Copilot systems often:
"go through multiple iterations to retrieve relevant pieces of information until it is satisfied that it has enough information, and then have another LLM call to synthesize the response."
Looking all these and considering (on assumptions)
For a simple chat model call:
Metric Typical Range TTFT 0.5–3 sec Short answer 1–10 sec Complex reasoning 5–20 sec For an agentic workflow (Copilot, Data Agent, RAG):
Step Latency Schema retrieval 1–5 sec Planning 1–5 sec SQL generation 1–5 sec Execution <1 sec on 4K rows Answer synthesis 2–10 sec Total: 10–30+ seconds is not unusual for multi-step agents even when the source table is tiny
Hope this gives a new prospective of looking into the evolution till we have the complete clearity
Thank you so much for this information
- ShivekMaharaj
Solution Sage
Hi vishal307088,
One small point on the Standard vs Preview test: I wouldn't expect Preview to necessarily be faster.
Microsoft's current Data Agent runtime documentation says both runtimes receive the same underlying LLM model upgrades. The difference is mainly in the orchestration, routing and query-generation components. Preview currently includes newer capabilities such as Advanced NL2SQL/DAX, some of which can actually use additional reasoning steps to improve accuracy.
So the similar ~29s result between Standard and Preview isn't too surprising.
Also, the 4,000-row table size only affects one part of the request. The Data Agent processing flow includes schema/context retrieval, source selection, query generation and validation, execution, and final response construction.
I would expand Show steps, take the generated SQL/DAX and run it directly. If that completes almost immediately while the full Data Agent request consistently remains around 29s, that gives you fairly strong evidence that the time is outside the actual data-query execution.
I still haven't seen Microsoft publish a response-time SLA that would make ~29s an official expected baseline.
AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.
Hi trivedisunita ShivekMaharaj v-kathullac
Thank you so much for giving your insight on this
I've made some changes to the Gold layer data and updated the agent instructions, but the Data Agent still takes at least 30 seconds to return a result.However, when I run the same SELECT query directly in the SQL analytics endpoint, it returns in sec.
