Production-grade Fabric data agents: moving from Demo to Dependable
Getting a Fabric data agent to work in a demo is one thing. Supporting it once people depend on it is another. Using the Lakeshore Retail demo, this guide covers the practical work behind that transition: reliable data, ontology context, secure access, performance testing, guardrails, and day-two operations. Tutorial data - Lakeshore Retail. The inventory is a CSV reference, not an agent answer; benchmarks are proposed, not measured. 1. Define the service before tuning the prompt Start with verifiable questions: inventory below safety stock, inbound shipments, governed sales measures, and refrigeration readings over a defined window. Agree on definitions, units, time zone, snapshot selection, freshness, and an answer owner. Low stock is not a shortage forecast. Two high-temperature readings do not prove a continuous breach: cadence, gaps, quality, and boundaries matter. Store-level relationships do not identify which product occupied a refrigeration unit. Put durable calculations in governed measures, SQL, or KQL. Ordering and other write actions need separate authorization and approval. Figure 1. Reference design; guardrails do not replace authorization. Use the standard runtime for production. Ontology, preview runtime, Foundry integrations/agent guardrails, evaluation SDK, and data-agent source control have separate preview considerations. The core's GA status does not extend to every dependency. 2. Use ontology for meaning, not as a security boundary LakeshoreOntology is now attached to the LakeshoreDataAgent draft, with Standard selected. It supplies entities, relationships, and mappings. SQL, KQL, or DAX executes against underlying sources under user permissions. Figure 2. Tutorial ontology, not proof of query correctness. The integration is preview. Attachment here does not confirm a service-wide fix to the documented new-experience issue. Keep shared definitions in the ontology. Explicitly mount eligible underlying sources before adding source-specific instructions, descriptions, or examples. These settings do not write back to the ontology. 3. Establish one reference answer For the fixed 2026-08-28T18:50:00Z snapshot, select positions where: LocationType = STORE StorageClass = FROZEN OnHandQuantity < SafetyStockQuantity The 48-position inventory fixture has exactly three matches, all Vanilla Bean Ice Cream (P-FRZ-001): Figure 3. A calculation from tutorial files, not a live agent result. For test INV-001, assert exact Store/Product keys, count, quantities, filters, and timestamp, not just three city names. The fixture's InventoryStatus = 1 marks low stock; sample instructions call it healthy. Resolve this first; the reference uses quantities. Do not silently alter data or treat August as current. 4. Prepare and wire the sources Curate objects without losing keys, joins, dates, or measures. Smaller is not automatically better. Use governed Power BI measures, Prep for AI, and Verified Answers. DAX generation uses model metadata and Prep for AI, not agent instructions. SQL/KQL source instructions/examples are separate. AI scoping and hidden fields are not authorization. Column flags do not narrow a selected source table; use supported model configuration or curated objects, not destructive edits to shared models. In development: Check prerequisites: supported paid F2+/P1+ capacity, tenant settings, and same-region agent/source capacities. Use Sources > Add sources > Add an ontology to attach LakeshoreOntology. Review its entities and Ontology data sources. Mount eligible sources for SQL/KQL configuration; prepare the semantic model separately. Set routing, clarification, and response rules, including units, identifiers, and honest failures. Inspect the fixed inventory question's source, generated query, and result. Publish to a restricted test audience; repeat as real consumers. Figure 4. The supplied attached-source draft. It does not establish query, security, or performance success. Validate the published version; republish runtime changes. Standard does not freeze service-managed LLM upgrades. Model Read permits semantic-model queries; authoring needs broader rights. 5. Benchmark the whole request Schema width and preview runtime are experiments, not guaranteed gains. Measure authentication, planning, query generation, source execution, response generation, and wrapper/guardrail work. Fast engine execution does not establish that capacity will fix conversation latency. Figure 5. Proposed experiments, not benchmark results. Vary one factor before combinations: Import, Direct Lake, and DirectQuery have no universal speed ranking. Filter telemetry early. Advanced DAX value indexing currently needs Q&A enabled. Fix questions, data, identities, and versions. Separate new chats/follow-ups and cold/warm runs; interleave baseline/candidates. Never clear shared production caches for a test. Use approved synthetic load with representative volume, cardinality, skew, joins, and freshness; tutorial data cannot prove sizing. Record sample size, p50/p95, timeouts, failures, throttles, correctness, and resources. Count all attempted eligible requests when reporting correct completion within budget, including timeouts. Leave caching off until permission-aware keys, freshness, expiration, and revocation invalidation are proven. 6. Add Foundry without moving the authorization boundary A prompt/hosted agent can use Fabric for orchestration, safety, and escalation, not repair measures, grant access, or replace Fabric's query-generation model. Use defense in depth: Application: authenticate, allowlist tools/arguments, bound calls/concurrency/deadlines, and validate available structured results. Foundry: supported safety controls and orchestration. Fabric and sources: enforce the identity's permissions. Data/semantic layers: business meaning. Use delegated end-user identity for interactive access; test two differently entitled consumers through the whole route. Never silently use the owner's identity. The Fabric tool and Fabric IQ integration differ; the former requires the same tenant. Deployment-principal support does not establish query-principal support. The public published-agent MCP pattern is: https://api.fabric.microsoft.com/v1/mcp/workspaces/{workspaceId}/dataagents/{dataAgentId}/agent Verify OAuth, consent, and networking; private-link routes may need workspace-specific hosts. Keep tokens out of code/logs. Moderation is route-specific, not automatic for arbitrary MCP bridges or nested calls. Test input, tool calls, retrieved content, and output on your route. Safety approval does not prove accuracy or authorization. For hosted guardrails, prove filtering with negative tests. invocations needs invocations_moderation; invocations_ws has no content-safety moderation. Agent-level groundedness/spotlighting are unsupported. Implement and test any missing required control, or block release. Review Code Interpreter/action tools for isolation, retention, and approval. Exclude preview dependencies if policy requires it. 7. Test behavior, not convincing prose Use approved references and holdouts separate from tuning. Assert numbers, dates, filters, and authorized scope deterministically. Test ambiguity, no data, outages, expired identities, follow-up filters, threshold boundaries, gaps, injection, denied fields, and cross-user isolation. Check totals/errors for disclosures too. Test consumers: workspace Admin, Member, and Contributor bypass RLS. Admin success is not an RLS pass. The conversational cap is 25 rows by 25 columns. Compute totals in the source, not from truncated output; offer an authorized report/export route for complete datasets. The preview evaluation SDK uses question/expected_answer columns and a published target. Review detailed steps alongside assertions. Judge scores cannot override failed access checks. Example targets: all critical cases pass; broader correctness >=95%; no disclosures observed in the negative suite; 95% of attempted eligible requests correct within 30 seconds at five concurrent users. These are proposals, not measurements or guarantees. 8. Release a service someone can operate Figure 6. Proposed release gates, not an existing pipeline. Separate dev/test/prod. Version configurations, ontology/models, AI preparation, examples, policies, code, and evaluations. Data-agent source control is preview. Verify target bindings, permissions, region, and freshness. Publish and smoke-test consumers before expanding access. Practice rollback to a known-good release; it does not necessarily restore data, permissions, tools, or the service-managed model. Regress after service changes. Assign business, data, application, security, and operations owners. Monitor outcomes, p95, throttling, freshness, quality, and cost with correlation IDs/versions. Restrict trace access and retention; never log tokens or unrestricted results. Bound transient retries, honor Retry-After, and never retry denials as outages. Add deadlines, circuit breakers, and escalation. Distinguish no matches, no access, and failed queries. Review compliance/geographic boundaries for Foundry, external services, logs, and exports. Fabric's up-to-28-day history retention does not define external retention. Respect Purview/source policies. AI consumption includes instructions, examples, and history; source queries consume separately. Monitor AI Query and engines, budget Foundry/application costs, and compare cost per correct, authorized answer within target. Require evidence for scope, data quality, published bindings, consumer access, correctness, guardrails, load, rollback, ownership, and honest failures. Missing evidence blocks release. Dependable means useful within scope and supportable when things change.
