Self‑Learning Fabric Data Agent Using .pbip + SDK

Sometimes meaningful ideas emerge when you simply connect the dots from your past explorations.

A few months ago, I explored two completely different areas:

1️⃣ Building Fabric Data Agents using GitHub Copilot Agent Mode—bringing down the creation time from 1–2 days to under an hour.

Fabric Data Agent with GitHub Copilot Agent Mode - Microsoft Fabric Community

2️⃣ Understanding Power BI’s new .pbip format—a discovery that changed how I think about dashboard development.

Revolutionizing Power BI Development: Create Dashb... - Microsoft Fabric Community

What I didn’t realize back then was that these two explorations were quietly preparing the ground for something far more powerful.

This article is where everything converges.

I’ll share how I combined both discoveries to build production‑ready, self‑learning Fabric Data Agents—where:

• .pbip files become a validated knowledge base for agent creation
• Existing Power BI reports act as ground truth for automated accuracy testing

💡 The breakthrough moment: when I realized that my Power BI report’s JSON definition could teach my Data Agent how to answer questions correctly.

⏱️ Before:Days of manual testing and guesswork

⚡ Now: Automated accuracy validation in minutes

🔁Result: A continuous learning loop that improves itself over time

📦 Sample Code Repository

All code discussed in this article is available on GitHub:

🔗 https://github.com/harigouthami/fabric-data-agent-accuracy-framework

The repository includes:

• 📓 6 modular Fabric notebooks (Setup → Configure → Examples → Query → Accuracy → Self‑Learning)
• 📊 Sample data (anonymized usage metrics)
• ⚙️ Configuration templates
• 🛠️ Utility functions (SQL validator, PBIP extractor)

💡 Tip: Clone the repo and upload the notebooks directly into your Fabric workspace to get started.

✨ A Personal Journey: When Two Explorations Became One

Let me share the moment when everything finally connected.

🔍 Exploration 1: Fabric Data Agent + GitHub Copilot

Creating a Data Agent in under an hour felt almost unreal. GitHub Copilot helped me:

• Automatically explore schemas
• Generate AI instructions
• Build example queries with minimal effort

But one question kept bothering me:

“How do I know the agent’s answers are actually correct?”

🎨 Exploration 2: Power BI .pbip + GitHub Copilot

Then came the .pbip discovery.

The realization that Power BI reports are simply structured JSON files felt like unlocking a hidden door. With Copilot, I could:

• Read visual configurations programmatically
• Understand DAX measures and relationships
• Generate visuals and models entirely through code

And then the thought that changed everything:

“If both my report and my agent are JSON… why can’t they learn from each other?”

🎯 The Convergence: This Article

One evening, while browsing through a .pbip folder, it all clicked:

My Power BI Report (.pbip)

├── definition/

│ ├── report.json ← Visual definitions

│ └── model.tmdl ← DAX measures & relationships

└── reportExtensions.json ← Custom logic

This wasn’t just a report. It was a business‑validated knowledge base.

Those DAX measures already represented:

• Approved calculations
• Trusted metric definitions
• Logic verified by stakeholders

If my Data Agent’s SQL produced the same results as those DAX measures, I could be confident in its accuracy.

That’s when I literally jumped out of my chair. 🎉

📘 From Prototype to Production: The Real Challenge

Once the initial excitement faded, real‑world questions surfaced:

• How do I systematically validate agent responses?
• How do I compare them against existing Power BI reports?
• How do I improve accuracy continuously—without manual review?
• How do I manage the agent programmatically, not through clicks?

The answer turned out to be a modular notebook framework powered by the Fabric Data Agent SDK.

🧠 Why the Fabric Data Agent SDK?

The SDK unlocks a fully code‑driven lifecycle:

• ✔ Programmatic agent creation and configuration
• ✔ Few‑shot example management via code
• ✔ Query execution and response validation
• ✔ Automated publishing and updates
• ✔ Seamless CI/CD integration

Instead of clicking through the Fabric UI, everything becomes version‑controlled, reproducible, and automatable.

📐 Unified Architecture: The Knowledge Triangle

At the heart of this approach is what I call the Knowledge Triangle:

• .pbip reports → trusted business logic
• Power BI DAX → ground truth
• Data Agent SQL → AI‑generated responses

When all three agree, confidence becomes measurable—not assumed.

🎯 Accuracy Testing: Where Confidence Comes From

Instead of manually checking answers, I compare:

• 🤖 Agent SQL output
• 📊 Power BI DAX result

When both match, the agent earns its confidence.

🔄 Self‑Learning: Closing the Loop

Failures are no longer setbacks—they’re training data.

When a test fails:

1. Capture the question
2. Identify the correct DAX
3. Generate validated SQL
4. Add it as a new example
5. Re‑publish the agent

This creates a continuous improvement loop that learns automatically.

💡 The Bigger Picture: Three Articles, One Story

What started as curiosity turned into a framework.

🏁 Final Thoughts: When Everything Clicks

The moment I realized that my .pbip files could:

1. Train my Data Agent
2. Validate its accuracy
3. Improve it continuously

…was one of those rare “everything aligns” moments.

My advice if you’re building Data Agents:

Don’t start from scratch if you have reports already available.

Your Power BI reports already contain:

• ✅ Verified business logic
• ✅ Trusted calculations
• ✅ Approved metric definitions

They are the best teachers your Data Agents can have.

🔮 What’s Next?

I’m now exploring:

• 📌 Multi‑Report Agents (one agent → multiple .pbip sources)
• 🔧 CI/CD for Agents (auto‑deploy when .pbip changes)
• 🧠 Semantic Layer Sync (agent & report always aligned)

Have you discovered interesting connections within Fabric? I’d love to hear your thoughts! 💬