Forum Discussion
Data Agent offer different numbers in Fabric Data Agent vs same agent in Copilot 365
I have created the data agent in Fabric and publish it to Copilot 365. I notice, in most cases, these two UIs give me totally different numbers to the same question I asked. Any fix on this?
2 Replies
- apturlov
Super User
Hello, Jeanxyz. Unfortunately without additional information that shows examples of such behavior it's hard to suggest concrete fixes. But, most likely, you don't have precise instructions in your Fabric Data agent that limit possible answers. The issue is, most likely, that when you use Copilot 365 the question is interpreted by the Copilot 365 and then the Data Agent is accessed not via its UI but another way, through the published endpoint, that is programmatically. This is multi-agent scenario. Then, Copilot creates its own answer from the Data Agent response. You are dealing with a broken phone scenario. You need to customize and thoroughly test instructions in your Data Agent that make sure that programmatic access follows the same path of interpretation as the UI chat. I cannot give you exact instructions, but I hope you get an idea. The most reliable way is to have query examples in your Data Agents instructions associated with keywords or topics in the questions that the Data Agent receives.
If you share some samples of behavior it might be possible to think of specific ways to formulate the instructions.
Think of it this way: two data engineers receive the same vague request and each one creates their own version of a query. Occasionally they may give the same results, but most likely they will provide different results, unless both of them follow the same precise pattern of querying the data.Hope this helps unblock your journey.
- arabalca
Super User
Hi Jeanxyz
Before jumping to conclusions, it's worth ruling out a few things, because the cause could be in several places.
The first thing to keep in mind is that the underlying architecture is not identical. When you query the Fabric data agent from the Fabric UI itself, it relies on the Azure OpenAI Assistant APIs within the Fabric boundary (more https://learn.microsoft.com/en-us/fabric/data-science/concept-data-agent). However, when you consume it from outside Fabric (published to Teams, M365 Copilot or Copilot Studio), the question first goes through the M365 Copilot orchestrator, which rephrases the question and reasons over the results before returning the final answer. This is officially documented and, according to Microsoft, "some level of change is inevitable" even if you add instructions to deliver the output as-is: https://learn.microsoft.com/en-us/fabric/data-science/data-agent-microsoft-365-copilot
Some variation in natural language is expected. Numbers changing, however, shouldn't happen if the underlying query is the same and the user has the same permissions.
To help you better, could you share a bit more?
- Are you testing with the same user in both UIs, or different accounts? If they differ, it could be a permissions issue on the data source (lakehouse, warehouse, semantic model, etc.), RLS/OLS applied, or even access to the data agent itself.
- Have you compared the query the agent generates in each case? In the Fabric UI you can see the SQL/DAX generated in the agent steps. If the query is the same and the raw result matches, the issue is in how the M365 Copilot orchestrator rewrites the output. If the query is already different, then M365 Copilot is rephrasing the question before passing it to the agent.
- What kind of source does the data agent use (warehouse, lakehouse, semantic model, KQL...)? In semantic models, if you have DAX measures with very similar names, the agent might "pick" a similar but not the same measure, and that gives you different numbers even though the question looks identical. In that case, it's worth tightening the agent's instructions to be much more restrictive: explicitly state which measure to use for each type of question, add examples in the Q&A pairs, and if needed, disambiguate by full name.
- Are the questions you're comparing literally the same word by word, or rephrased?
- Have you tried adding instructions in the publish description such as "return numeric values as-is, do not summarize or recompute"? That's the official way to minimize variation.
With that info we can focus the diagnosis better. Hope these tips help you solve or at least clarify the issue.
If this helped, please consider giving it a Like. If it solved your issue, please mark it as the Accepted Solution to help others facing the same problem.
Thanks