Fabric Data Agent answers wrongly through SDK but not through UI

[luisgt97]

Hi there!

I have a python backend that works as an assistant to our sales clients, based on our Fabric Data Agent. I used this doc how to consume FDA through SDK.

It works well enough (NLP -> DAX query process is quite a time bottleneck) but sometimes I encounter this issue:

When I ask the assistant through our backend about some data, it answers there has been an error or it does not know where this data is.

Then I go to Fabric UI and ask the same on the Fabric Data Agent tab itself. Then this time it knows how to answer propperly and finds the data. If I try to ask a second time from our backend, it then answers correctly using the same answer as in the UI.

Is this a known issue? Is there a way to solve this?

[luisgt97]

Hi @v-hjannapu 

And this built-in context could maybe be the Prep for AI data, Semantic model descriptions we wrote and so?
In this case, using it through API is almost as using a baseline model, with the only difference that it can perform DAX queries?

If this is the case, and we want to integrate our agent to our clients webpage, what would be the best way? Consume it through a Foundry AI Agent or a Copilot Agent? Right now, we tried to consume it on a copilot studio agent through MCP and it fails continously to apply the right filters to our data, resulting in 0 results answers and hallucinations. 

[v-hjannapu]

Hi @luisgt97,

Yes, you are correct. In Fabric UI, the agent makes use of the semantic model details and “Prep for AI” metadata, that’s why it gives better results there. When you call through SDK, that extra context is not always applied, so it behaves more like a base model that only runs DAX.

If you plan to connect this to your client’s webpage, the more reliable way is to consume it directly through the Fabric Data Agent API. Going through Copilot Studio or other layers can sometimes drop the grounding info, which leads to wrong filters or empty results.

Regards,
Harshitha.

[luisgt97]

I see, thanks @v-hjannapu !

but then just one question, how do I consume it through the FDA API from my client's webpage? Is there any doc to do it? I just found this doc , and it was just only realeased a week ago, to connect to the published FDA URL. This is what my backend is built around, and it is just not working propperly because of this lack of context...

Regards,

Luis.

[v-hjannapu · Accepted solution]

Hi @luisgt97,

You are right, the doc you shared is the latest one for consuming FDA through Python/SDK. At the moment, that is the supported way to connect to the published FDA URL. The difference you are seeing is because the SDK call doesn’t automatically bring in all the semantic model context like the Fabric UI does. That’s why the answers look inconsistent.

For your client webpage, you can continue to call the FDA endpoint the same way as shown in the doc, but you will need to make sure your prompt includes the right context for example, model descriptions or important filters. Right now, adding this context manually is the way to make the SDK behavior closer to the UI.

Regards,
Community Support Team.

[luisgt97]

Hi there again!

Seems that adding the system_prompt into instructions parameter of OpenAI Assistant constructor improves answers a little. At least, those that I tried and were being answered incorrectly. I will keep an eye on news from Fabric team on this matter. Thanks for your help!

[luisgt97]

EDIT 1:

Hey, figured I should update this with an edit as I know what is causing the error. In the FDA instructions, I have a part of the prompt forcing the agent to always filter DAX queries by todays date and manufacturer name, a variable of our semantic model feeding the FDA.

While sending API requests like this repo, FDA always hallucinates and tries to filter by store name. While using the UI, it never hallucinates and does it propperly. Any ideas why is this happening?

[v-hjannapu]

Hi @luisgt97,

Thank you  for reaching out to the Microsoft fabric community forum.

The difference you are seeing is mainly because in Fabric UI the Data Agent uses some built-in context, but when you call through SDK it fully depends on the way your prompt is written. That’s why the SDK sometimes interprets it differently and adds filters like Store Name.

You can try making the prompt a bit flexible instead of forcing it strictly, as softer instructions usually reduce this type of hallucination. Also, checking the request body you send through SDK against what UI generates may help you spot if something is missing.

Hope this helps you handle the difference.
Regards,
Harshitha.