Forum Discussion

Ashwath_Bala_S's avatar
4 months ago
Solved

Dataverse Data Live Integration with Foundry Agents via Fabric

Hello Team,

 

As part of developing Agents in Foundry, we have a few questions and would appreciate your inputs.

 

Currently, our primary data resides in Dataverse (with the source system being FnO). The data is connected to Microsoft Fabric using Fabric Link, and as a result, the Dataverse tables are available in Fabric as shortcut (linked) tables.

Our objective is to enable Foundry Agents to retrieve this data effectively.

In this context, we would like to understand the following:

  1. Is there a supported way to materialize Dataverse table data into Microsoft Fabric OneLake (for example, as Lakehouse tables or files), so that we can create a Search / Vector Index and connect it to a Foundry Agent?
  2. Are there any alternative or recommended approaches to connect Fabric data to Foundry Agents, especially when the source data originates in Dataverse?

At present, our data remains in Dataverse, and we are looking for guidance on:

  • Best practices to bring this data into Fabric (OneLake), and
  • The supported pattern to connect it to Foundry Agents for near real-time or frequently refreshed data retrieval.

We would appreciate your inputs and recommendations on the most suitable architecture for this scenario.

Thank you in advance for your support.

 

Regards,

Ashwath Bala S

  • Hola Ashwath_Bala_S 

     

    Thanks for sharing the scenario. I believe there are some important architectural considerations to take into account before choosing the final pattern.

    First, with Fabric Link, the data exposed in Fabric is not the same as building a fully materialized analytical layer from the start. In Finance and Operations scenarios, it is also important to distinguish between tables and entities, as both can be enabled and used differently.

    Second, it is important to carefully consider the synchronization behavior. According to Microsoft documentation, Fabric Link triggers refresh tasks every 15 minutes, and updated files may appear within approximately 15 to 45 minutes depending on volume and change frequency. Therefore, I consider this solution to be frequently updated, near real-time for analytical purposes, but not suitable for strict real-time ingestion.

    Based on this, my recommendation would be to use a medallion-style architecture:

    • Bronze: keep Fabric Link as the raw ingestion layer
    • Silver: materialize and normalize business data into physical tables in OneLake
    • Gold: expose curated, business-ready tables/views for agent consumption

    In practice, I would not recommend using the linked layer directly as the final source for Foundry configuration. It is generally better to first use structured and organized business objects, especially when the source comes from Dataverse/FnO entities.

    For the Silver and Gold layers, it is worth considering Materialized Lake Views. Microsoft documents them as a way to implement medallion architecture in Fabric Lakehouse, with results stored as Delta files in OneLake and automatically updated through schedules or event triggers. This helps reduce execution time and makes transformations more operational and reusable.

    From there, I see two compatible patterns depending on the target agent technology:

    • If the goal is conversational access to structured Fabric data: a Fabric Data Agent is a great option. These agents can use Lakehouses, Warehouses, Power BI semantic models, KQL databases, and ontologies as sources.
    • If the goal is specifically Azure AI Foundry Agent Service with retrieval/vectorization: the most compatible pattern currently seems to be: curated Fabric data → indexing layer → Azure AI Search / Foundry IQ knowledge base → Foundry Agent

    It would be advisable to assume that a Foundry agent can directly use a Fabric Data Agent as its knowledge source, as official Foundry guidance focuses on connections such as knowledge bases.

    In summary, my recommendation would be: F&O → Fabric Link (Bronze) → Materialized business tables (Silver) → Curated service layer (Gold) → Consumption by Fabric + Foundry agents (Data Agent knowledge)

    This provides:

    • separation between ingestion and consumption
    • better control over refresh and data quality
    • a more scalable and compatible pattern for agent integration
    • flexibility to use either Fabric Data Agents or Azure AI Foundry agents depending on the final use case

    If my comment helped you resolve your question, I would appreciate it if you marked it as the accepted solution and gave it a like. This helps other users with the same problem and motivates me to continue contributing.

    Thank you very much.

  • Hello Ashwath_Bala_S 

     

    You can’t query Lakehouse tables, files, or Dataverse shortcuts directly from Foundry Agents. The supported and intended pattern is Foundry Agent → Fabric Data Agent → Fabric data sources. Fabric Data Agent does the heavy lifting here – it provides the semantic layer, security enforcement, and query translation (NL to SQL, DAX, or KQL), while Foundry simply orchestrates and invokes tools rather than accessing data itself.

     

    For Dataverse specifically, the data must first sit behind a Fabric‑supported analytical surface before a Data Agent can use it. Using Fabric Link (Dataverse Link to Fabric) is the right starting point, as it exposes Dataverse data as linked Delta tables in OneLake. It’s worth remembering that Fabric Link syncs every 15–45 minutes, which Microsoft positions as near real‑time for analytics, not for operational or transactional use.

     

    You’ll generally get better results if you don’t expose those linked Dataverse tables directly to agents. A more robust approach is to apply a medallion architecture: keep linked tables as Bronze, materialise and clean them into Silver, then curate agent‑ready Gold tables or a semantic model. Fabric Data Agents perform noticeably better when querying clean, well‑modelled schemas rather than raw or loosely structured sources.

     

    Fabric Data Agents can sit on top of Power BI semantic models, curated Gold Lakehouse tables, or Warehouses. In practice, semantic model‑backed agents tend to give higher accuracy and shorter prompts because they can lean on rich relationships, measures, and metadata. If you also need search or vector capabilities, treat that as a complement: extract text from curated Fabric tables, build Azure AI Search or vector indexes, and expose that alongside the Fabric Data Agent as a separate tool in Foundry. That multi‑tool pattern extends what the agent can do, rather than replacing the Data Agent altogether.

     

5 Replies

  • Hola Ashwath_Bala_S 

     

    Thanks for sharing the scenario. I believe there are some important architectural considerations to take into account before choosing the final pattern.

    First, with Fabric Link, the data exposed in Fabric is not the same as building a fully materialized analytical layer from the start. In Finance and Operations scenarios, it is also important to distinguish between tables and entities, as both can be enabled and used differently.

    Second, it is important to carefully consider the synchronization behavior. According to Microsoft documentation, Fabric Link triggers refresh tasks every 15 minutes, and updated files may appear within approximately 15 to 45 minutes depending on volume and change frequency. Therefore, I consider this solution to be frequently updated, near real-time for analytical purposes, but not suitable for strict real-time ingestion.

    Based on this, my recommendation would be to use a medallion-style architecture:

    • Bronze: keep Fabric Link as the raw ingestion layer
    • Silver: materialize and normalize business data into physical tables in OneLake
    • Gold: expose curated, business-ready tables/views for agent consumption

    In practice, I would not recommend using the linked layer directly as the final source for Foundry configuration. It is generally better to first use structured and organized business objects, especially when the source comes from Dataverse/FnO entities.

    For the Silver and Gold layers, it is worth considering Materialized Lake Views. Microsoft documents them as a way to implement medallion architecture in Fabric Lakehouse, with results stored as Delta files in OneLake and automatically updated through schedules or event triggers. This helps reduce execution time and makes transformations more operational and reusable.

    From there, I see two compatible patterns depending on the target agent technology:

    • If the goal is conversational access to structured Fabric data: a Fabric Data Agent is a great option. These agents can use Lakehouses, Warehouses, Power BI semantic models, KQL databases, and ontologies as sources.
    • If the goal is specifically Azure AI Foundry Agent Service with retrieval/vectorization: the most compatible pattern currently seems to be: curated Fabric data → indexing layer → Azure AI Search / Foundry IQ knowledge base → Foundry Agent

    It would be advisable to assume that a Foundry agent can directly use a Fabric Data Agent as its knowledge source, as official Foundry guidance focuses on connections such as knowledge bases.

    In summary, my recommendation would be: F&O → Fabric Link (Bronze) → Materialized business tables (Silver) → Curated service layer (Gold) → Consumption by Fabric + Foundry agents (Data Agent knowledge)

    This provides:

    • separation between ingestion and consumption
    • better control over refresh and data quality
    • a more scalable and compatible pattern for agent integration
    • flexibility to use either Fabric Data Agents or Azure AI Foundry agents depending on the final use case

    If my comment helped you resolve your question, I would appreciate it if you marked it as the accepted solution and gave it a like. This helps other users with the same problem and motivates me to continue contributing.

    Thank you very much.

  • Hello Ashwath_Bala_S 

     

    You can’t query Lakehouse tables, files, or Dataverse shortcuts directly from Foundry Agents. The supported and intended pattern is Foundry Agent → Fabric Data Agent → Fabric data sources. Fabric Data Agent does the heavy lifting here – it provides the semantic layer, security enforcement, and query translation (NL to SQL, DAX, or KQL), while Foundry simply orchestrates and invokes tools rather than accessing data itself.

     

    For Dataverse specifically, the data must first sit behind a Fabric‑supported analytical surface before a Data Agent can use it. Using Fabric Link (Dataverse Link to Fabric) is the right starting point, as it exposes Dataverse data as linked Delta tables in OneLake. It’s worth remembering that Fabric Link syncs every 15–45 minutes, which Microsoft positions as near real‑time for analytics, not for operational or transactional use.

     

    You’ll generally get better results if you don’t expose those linked Dataverse tables directly to agents. A more robust approach is to apply a medallion architecture: keep linked tables as Bronze, materialise and clean them into Silver, then curate agent‑ready Gold tables or a semantic model. Fabric Data Agents perform noticeably better when querying clean, well‑modelled schemas rather than raw or loosely structured sources.

     

    Fabric Data Agents can sit on top of Power BI semantic models, curated Gold Lakehouse tables, or Warehouses. In practice, semantic model‑backed agents tend to give higher accuracy and shorter prompts because they can lean on rich relationships, measures, and metadata. If you also need search or vector capabilities, treat that as a complement: extract text from curated Fabric tables, build Azure AI Search or vector indexes, and expose that alongside the Fabric Data Agent as a separate tool in Foundry. That multi‑tool pattern extends what the agent can do, rather than replacing the Data Agent altogether.

     

  • v-priyankata's avatar
    v-priyankata
    Community Support

    Hi Ashwath_Bala_S 

    Thank you for reaching out to the Microsoft Fabric Forum Community.

    arabalca deborshi_nag Thanks for the inputs

    I hope the information provided by users was helpful. If you still have questions, please don't hesitate to reach out to the community.

     

    • v-priyankata's avatar
      v-priyankata
      Community Support

      Hi Ashwath_Bala_S 

      Hope everything’s going smoothly on your end. I wanted to check if the issue got sorted. if you have any other issues please reach community.

  • deborshi_nag hit the nail on the head here. Trying to point Foundry Agents directly at raw Lakehouse tables or relying strictly on standard Fabric Link syncs usually leads to massive prompt bloating, high token costs, and agents hallucinating.

    We run an AI automation agency , and we run into this exact architectural bottleneck all the time when building enterprise agents. Based on what we’ve seen deployed in production, here are a few practical insights on how to actually execute this without breaking the agent's logic:

    1. Treat the Semantic Layer as the Agent's "Brain"

    As Deborshi mentioned, agents perform significantly better on a curated Gold semantic model. The agent shouldn't have to figure out what an "active client" or "monthly revenue" means by looking at raw schemas. In our projects, we always make sure all business logic and DAX measures are tightly packaged before the agent touches them. This keeps prompts incredibly short and pushes accuracy close to 100%.

    2. Watch out for the 15-45 Minute Sync Bottleneck

    Fabric Link’s sync window is flawless for analytical dashboards, but it can be a massive operational bottleneck. If your Foundry Agent needs to act on a fresh CRM update right now, waiting 30 minutes is a dealbreaker. To solve this for our clients, we usually set up event-driven webhooks. This allows the agent to fetch critical, real-time operational data via direct Dataverse API calls on the fly, while still leveraging Fabric for heavy historical queries.

    3. Mastering the Multi-Tool / Hybrid RAG Pattern

    If your agent needs to check structural CRM data and parse unstructured text (like service contracts or PDFs), you will definitely need that multi-tool architecture combining Fabric Data Agents with Azure AI Search or a custom vector index. Orchestrating these so the agent intelligently switches between SQL/DAX querying and semantic vector search requires a lot of API fine-tuning, but it’s the only way to build a robust system.

    It’s a complex stack, but building that automated bridge between your data governance and the AI orchestration layer is game-changing. Let me know if you run into any issues with the pipeline mapping happy to share what we've learned from our setups!