Conversational Analytics on Fabric: Fabric Data Agent vs. Custom Azure AI Architecture with Copilot

Hello twissamodi,

This is a great architectural question, and I don't think there is a single recommended architecture for every scenario.

The choice depends on your governance requirements, scale, customization needs, and how much semantic intelligence you want Microsoft Fabric to provide versus building it yourself.

A few considerations:

1. Fabric-native semantic approach (Semantic models, Fabric Data Agent, Power BI MCP Server)

If your users primarily need governed business analytics over curated data, I would evaluate the Fabric-native capabilities first.

Semantic models already contain measures, relationships, hierarchies, descriptions, and can include synonyms and other business metadata, reducing the amount of context that must be recreated elsewhere.

From a governance perspective, this also centralizes business logic instead of duplicating it across custom services.

2. Custom Azure AI architecture

A custom Azure AI service provides greater flexibility, especially if you need orchestration across multiple systems, custom prompting, external APIs, or non-Fabric data sources.

However, it also introduces additional components to maintain, such as:

• prompt orchestration
• business metadata management
• security trimming
• caching
• monitoring
• lifecycle management

Those responsibilities are largely yours rather than being provided by the Fabric semantic layer.

3. Security and permissions

For semantic-model-based solutions, access is still governed by Fabric and Power BI permissions. Users or the calling identity must have the appropriate permissions to access the semantic model, and Build permission is required for some scenarios involving semantic model reuse or live connections.

The exact permission model depends on the Microsoft capability being used (for example, Copilot, Fabric Data Agent, or Power BI MCP Server), so I would validate the licensing and permission requirements for the chosen component before finalizing the architecture.

4. Business metadata

If building a custom Azure AI solution, I would avoid duplicating business definitions in prompts wherever possible.

Instead, I would keep business terminology, measures, relationships, descriptions, and governance within Fabric semantic models where possible, and expose that governed semantic layer to the AI application. This reduces the risk of inconsistent business definitions over time.

5. Scale

With approximately 2,500 active users, I would also consider:

• Fabric capacity sizing
• semantic model concurrency
• Copilot licensing
• security model
• operational monitoring
• governance
• long-term maintenance

These factors may have a greater impact on the architecture than the conversational interface itself.

Overall, if the primary use case is governed business analytics over Fabric data, I would start by evaluating the Fabric-native semantic capabilities before introducing a custom Azure AI layer.

If additional orchestration, external knowledge, or complex workflows are required, then a hybrid architecture may provide the best balance between governance and flexibility.

Microsoft references:

• Microsoft Fabric Data Agent: https://learn.microsoft.com/fabric/data-science/data-agent
• Power BI semantic models: https://learn.microsoft.com/power-bi/connect-data/service-datasets-overview
• Copilot Studio documentation: https://learn.microsoft.com/microsoft-copilot-studio/
• Microsoft Fabric architecture guidance: https://learn.microsoft.com/fabric/fundamentals/

I'd also be interested to hear from anyone who has deployed a similar solution at enterprise scale, particularly around licensing, semantic model permissions, and operational experience.

Best regards,
Omkar Shinde
Microsoft Fabric Enthusiast | Power BI Consultant

💡If you found this response helpful, please consider giving it a Kudos.
✅If this resolves your question, please mark it as the Accepted Solution to help others in the community.