Data Engineering Community Blog articles

Kimball or Data Vault? How to select your Data Model that can be Optimised for your Business?

Vinodh247 — Tue, 10 Nov 2026 19:33:27 GMT

Choosing between Kimball and Data Vault is often framed as a contest between two modelling styles. But they solve different problems. The more useful question is: what does your data platform need to optimise for?

Kimball: Make analysis easier

Kimball dimensional modelling structures data around business processes, facts and dimensions. Denormalised dimensions help analysts understand the data and write straightforward queries.

Surrogate keys, often integers, identify dimension records and support the tracking of changes over time. When a dimension’s history matters, a fact should reference the version that was valid when the business event occurred.

Kimball also supports enterprise-wide consistency through conformed dimensions and the enterprise bus architecture. Its strength is bringing data closer to the people who use it: reporting is intuitive, joins are easier to understand, and business-facing solutions can be delivered quickly.

Data Vault: Make integration and change easier

Data Vault separates the model into three core structures:

Hubs - store business keys.
Links - capture relationships between business entities.
Satellites - hold descriptive attributes and their history.

Deterministic hash keys can reduce reliance on centrally generated sequence keys, supporting parallel ingestion across sources. The approach is designed to preserve source-aligned history, improve traceability & adapt as sources change.

That flexibility comes with a trade-off. A Data Vault typically involves more tables, orchestration and complexity between raw data and analyst-ready data. Business Vault logic, Point-in-Time tables, bridges and dimensional marts may be needed to make the data convenient for consumption.

Hash keys also depend on a shared, consistently applied standard. Pipelines need to agree on details such as casing, whitespace, null handling, delimiters, column order, encoding and source scope. Without that agreement, the same business key can produce different hashes.

Often, the answer is both

Kimball optimises for analytical consumption. Data Vault optimises for integration, traceability and resilience to change. These priorities are different, but they are not mutually exclusive.

Many enterprise platforms use a vault to preserve source-aligned history and integrate evolving systems, then build dimensional models to provide a governed, query-friendly view for analysis.

For a new Fabric platform, the decision should reflect the realities of the project: source complexity, audit and lineage requirements, delivery speed, team maturity and the simplicity analysts need. The best model is not the one that is fashionable; it is the one that fits the responsibility your data architecture must fulfil.

Where Data Meets AI Extending Azure Databricks Investments with Microsoft Fabric

AnujPandey — Thu, 05 Nov 2026 14:45:28 GMT

When organizations discuss enterprise AI, the conversation almost always begins with foundation models, prompts, and agents. Yet in practice, every production enterprise AI initiative succeeds or fails based on how well it answers three equally critical questions:

Where does the trusted enterprise data come from?
How does business intelligence and context get created from that data?
Where will business users ultimately experience and consume that intelligence?

This is the modern Azure journey: Data → Intelligence → Experience.

And at the very start of this journey sits a foundational architectural principle that guides everything in this post:

Modernization does not mean forced migration.
Don't change a customer's existing Azure Databricks architecture just to introduce Microsoft Fabric or AI. Instead, demonstrate what additional Microsoft capabilities can be layered around their existing investment with zero data duplication.

Why "Why Would We Change Our Architecture?" Is the Right Question

If an enterprise has spent years building curated Medallion architectures (Bronze, Silver, Gold), Lakeflow / Delta Live Tables (DLT) pipelines, and fine-grained governance in Azure Databricks Unity Catalog, asking them to rip-and-replace or duplicate pipelines just to light up Power BI Direct Lake or Microsoft Foundry agents is a non-starter.

A modern lakehouse strategy shouldn't say: "Here is how to leave Databricks."
Instead, it should prove: "Here is how an organization already invested in Azure Databricks can extend that architecture with Microsoft Fabric, OneLake, and Microsoft Foundry—while minimizing architectural disruption and avoiding unnecessary data movement."

Data Is 1/3 of the Story: The 3-Act Framework

As we designed our end-to-end reference implementation, we structured the architecture around three distinct acts. Your Databricks and Fabric connectivity foundation is Act 1—the essential 1/3 that unlocks the entire downstream AI and user experience:

https://raw.githubusercontent.com/anujpandey2/Databricks-Blogs/2fda4c01be139dd54742bb9d9af90593a7c106c4/images/3-act-architecture-diagram.svg

Act	Pillar	Guiding Principle	Key Question Answered	Core Technologies
Act 1 (1/3)	DATA	Meet customers where their data already lives.	Where is my enterprise data, and how can the broader Microsoft ecosystem work with it without copying it?	Azure Databricks, Unity Catalog, Mirrored Catalog, Microsoft Fabric, OneLake
Act 2 (2/3)	INTELLIGENCE	Turn trusted business data into intelligence.	How can AI models and agents reason over governed enterprise ontologies and metrics rather than raw schemas?	Fabric IQ (Ontology, Graph & Semantic Models), Fabric Data Agents, Microsoft Foundry, Azure OpenAI
Act 3 (3/3)	EXPERIENCE	Bring intelligence to where work happens.	Where and how will business users actually consume this intelligence without knowing the underlying plumbing?	Microsoft 365 Copilot, Microsoft Teams, Power BI, Custom APIs/Apps

State of the Union (As of November 2026): Azure Databricks & Microsoft Fabric Connectivity

As of late 2026, Azure Databricks and Microsoft Fabric interoperability has matured into a true bidirectional, zero-copy ecosystem built on open standards (Delta Lake, Apache Iceberg, and Unity Catalog Open APIs).

Here is the complete landscape of how the two platforms connect today:

1. Inbound to Fabric: Mirroring Azure Databricks Unity Catalog (Generally Available)

How it works: Fabric connects directly to your Azure Databricks Unity Catalog metastore using the Unity Catalog Open APIs. It creates a Mirrored Azure Databricks catalog item in Fabric and automatically generates OneLake shortcuts (Databricks type shortcuts) pointing to the underlying ADLS Gen2 Delta tables.
Zero-Copy & Auto-Sync: No data is moved or duplicated. When schemas, tables, materialized views, or streaming tables are added or dropped in Databricks Unity Catalog, Fabric automatically synchronizes the metadata (typically propagating changes within seconds to a few minutes).
Streaming Tables & Materialized Views Support: By enabling TBLPROPERTIES ('pipelines.externalMetadata.enabled' = 'true') on your DLT / Lakeflow pipelines or standalone objects, even Databricks Materialized Views and Streaming Tables can be mirrored directly into Fabric.
Built-in Consumption: Every Mirrored Azure Databricks catalog item automatically provisions a SQL Analytics Endpoint (for T-SQL queries, views, and warehousing), supports OneLake table shortcuts into Fabric Lakehouses, and powers Power BI Direct Lake semantic models and Fabric Data Agents.

2. Outbound from Fabric: OneLake Catalog Federation in Azure Databricks (Preview / Beta)

How it works: Going the other direction, Azure Databricks Lakehouse Federation now natively supports OneLake Catalog Federation (CREATE CONNECTION ... TYPE onelake and CREATE FOREIGN CATALOG).
Zero-Copy Access from Databricks Compute: Data engineers and data scientists running on Databricks Runtime 18.0+ (or DBR 19.0+ using workspace-specific endpoints <workspace-id>.z<xy>.dfs.fabric.microsoft.com) and SQL Warehouses (2025.40+) can query Fabric Lakehouse and Warehouse tables directly using standard three-part Unity Catalog names (fabric_catalog.schema.table) with zero data copying.
Unstructured Data Volume Support: Setting create_volume_for_lakehouse_files 'true' automatically mounts the Fabric Lakehouse /Files directory as a read-only Unity Catalog Volume (/Volumes/<catalog-name>/onelake-folders/files/) for Spark binary/Parquet/PDF processing in Databricks.

3. Shared Storage: OneLake & ADLS Gen2 Shortcuts

Both platforms can continue to read/write to existing ADLS Gen2 storage accounts, or Databricks can mount/read/write directly to OneLake endpoints (abfss://<workspace>@onelake.dfs.fabric.microsoft.com/) using Entra ID Service Principals, Managed Identities (via Access Connectors for Azure Databricks), or credential passthrough.

What Our E2E Interoperability Implementation Proves (The 5 Architectural Proofs)

To make this concrete—following the philosophy of Build Once → Prove It → Document It → Enable Others—our end-to-end walkthrough is designed to prove five non-negotiable enterprise requirements:

Existing Databricks investment remains meaningful: We start with a realistic Azure Databricks environment where data engineers curate Bronze, Silver, and Gold tables via Delta Live Tables (DLT) / Lakeflow pipelines registered in Unity Catalog.
Interoperability is real (not slideware): We configure live Unity Catalog Mirroring into Fabric and OneLake Catalog Federation back into Databricks.
Unnecessary data movement is eliminated: We explicitly show what stays in Azure Databricks and ADLS Gen2 (100% of the physical Parquet/Delta files) versus what gets federated into Fabric (metadata and zero-copy pointers).
Fabric layers immediate business value: Rather than stopping at "I can see my Databricks table in Fabric," we layer a Power BI Direct Lake semantic model, Copilot in Power BI, and a Fabric Data Agent on top of the mirrored Databricks tables.
Governance and security boundaries are preserved: We document exact authentication, network firewall rules (Workspace Identity + Trusted Workspace Access), and where Unity Catalog vs. OneLake security policies are enforced.

End-to-End Walkthrough: One Business Scenario, Multiple Personas

Let's walk through a unified enterprise scenario across four personas—Data Engineer, Analytics Engineer / BI Analyst, AI Developer, and Business User—answering a single executive business question:

"Why are sales declining and customer churn rising in our target segments, and which products and retention actions should we prioritize?"

Act 1 — DATA: Extending the Existing Azure Databricks Estate into Microsoft Fabric

Step 1: Curate Enterprise Data in Azure Databricks (Data Engineer Persona)

Our data engineering team already runs its core ingestion and Medallion transformations in Azure Databricks. We do not rewrite these pipelines in Fabric.

From the Azure Portal, we launch our existing Unity Catalog-enabled Azure Databricks workspace and sign in with Microsoft Entra ID:

Please refer to this Github link : anujpandey2/Databricks-Blogs: The Modern Azure Journey: Where Data Meets AI — Azure Databricks & Microsoft Fabric Interoperability Blog and Reference Architecture

Unified Data Access Pattern Across Fabric

apturlov — Tue, 03 Nov 2026 14:34:14 GMT

Have you ever seen a situation when in an ETL pipeline engineers create a not strictly necessary dataset only because they can use a tool such as a Spark or SQL notebook to process data in that dataset? I have and I don't think this is a best practice when data is replicated just to satisfy a tool selection preference. In this post I'll explain how Microsoft Fabric solves this problem by providing a unified way to access data in various formats by using Lakehouse table shortcuts.

Storage Choice Should Not Dictate the Exploration Tool

What is a Lakehouse table shortcut? You can read the official definition here(https://learn.microsoft.com/en-us/fabric/onelake/onelake-shortcuts), but a short explanation is they are very similar to Windows file shortcuts and allow accessing various tabular data sources in Fabric in a universal way as if they were Lakehouse delta tables.

Just imagine sources like SQL database or Warehouse tables and KQL tables, which all require different native tools to work with. A table shortcut makes various sources stored on OneLake in Delta format appear in a Lakehouse as delta tables and immediately accessible from a Spark Notebook or a SQL script via a Lakehouse SQL endpoint.

This is not a magic trick, and a table shortcut does not copy data behind the scene. When you create a table shortcut in a Lakehouse data is not duplicated but instead a new reading path becomes available through a Lakehouse API and associated SQL endpoint. A table shortcut looks like a Lakehouse delta table and can be accessed with the same tools, though it still points to the source data, and you can use your familiar tool of choice to work with that data source.

In this article I'll cover table shortcuts only. There are also File shortcuts in Fabric but they are a different case.

The Tree Decision Criteria

Now that you understand the concept of the Table shortcuts, how do you decide if you want to use a Lakehouse table shortcut for a particular data source that you need to explore? I start from answering these three questions:

Is the data source format and location supported by a table shortcut?
Will a Spark or T-SQL Notebook be a sensible choice for this data source exploration scenario?
Are there any hard road blocks such as permissions, latency, or performance for using a table shortcut?

If I pass on all three questions my choice is clear: create a table shortcut and start exploring the data.
Now let's talk about a few scenarios where a Lakehouse table shortcut could be a useful choice.

1. SQL Database and Warehouse

From the first glance it's not obvious why you would want to use a table shortcut for a SQL database or a Warehouse. Both already support exploration with T-SQL, so the shortcut has to earn its place.
For me, the strongest reason is that the work needs something that T-SQL does not do well, usually Python-based exploration or processing. Reusing existing PySpark code or working with the data as DataFrames next to other Lakehouse tables fall in the same category.

Why I Use It

Explore the data with PySpark without creating a duplicate table first.
Profile or transform the data before deciding what a curated result should look like.

Where It Shines

Exploratory analysis in Python.
Data-quality checks that are easier to express in code.
Data shaping for a larger Spark workflow.
Complex joins and transformations that combine SQL source data with native Lakehouse tables in Spark.

What to Keep in Mind

A SQL database and a Warehouse are not the same kind of source. A SQL database is transactional,
but its data is mirrored to OneLake as Delta files, so analytics reads against the mirrored copy don't load the primary workload. A Warehouse is built for analytical queries, so that concern is even smaller.
Treat shortcut-backed access as read-only for this pattern.
Shortcut behavior may change, so confirm against current documentation.

2. Mirrored SQL Database

A mirrored SQL database can also be queried with T-SQL via a SQL Endpoint, so it could belong in the first scenario. But I give it its own section because its role is different. A mirror is a projection of an external data source into OneLake not the actual data source such as SQL database or a Warehouse. This distinction makes it closer to a raw data landing area that requires ingestion first.

This is where using a shortcut gets exciting. A Lakehouse shortcut to the mirrored tables lets you read the data without running any ingestion workload. The shortcut removes the copy operations and brings the data into a bronze layer, from where it can be further cleaned or transformed as needed.

Why I Use It

Enable Spark and T-SQL to read replicated source data without another ingestion job.
Use the mirror as a starting point for initial data exploration.

Where It Shines

Early exploration of source data before the medallion design is settled.
Checking data quality against the source before any transformation.
Supplying data for a Silver layer for transformation without an initial ingestion.

What to Keep in Mind

A shortcut saves the copy operation, but downstream may still require their own tables.
The mirror reflects the current state of the source. Delta history is kept only for the retention period, which defaults to one day for newer mirrored databases, so materialize snapshots if you need longer history.
Replication is near real time, but latency depends on the source, region, and change volume. Check the current figures before promising freshness.

3. Lakehouse: The Simplest Data Sharing Case

A Lakehouse-to-Lakehouse shortcut is the easiest case. The team owning the data keeps the table where it is, and another team creates a table shortcut in their own Lakehouse. Both teams work with the same Delta table without creating a copy.

Why I Use It

Keep a table in the Lakehouse that owns it while another team uses it.
Join Delta tables from different Lakehouses via shortcuts in Spark or through the SQL analytics endpoint with simplified syntax.

Where It Shines

Cross-workspace analytics, where a central team needs a domain's curated table.
Domain-oriented ownership, where each team publishes its tables and others consume them.

What to Keep in Mind

A shortcut doesn't grant access. The calling user needs read permission on the target table.
Schema changes sync automatically, but timing can vary. A new shortcut may take time to show up.
Deleting a shortcut leaves the target untouched.
Shortcut names with spaces aren't recognized as Delta tables.
The workspace lineage view only shows shortcuts within the same workspace.

4. KQL Database: Same Goal, Different Mechanics

KQL Database in an Eventhouse isn't just another Delta table shortcut. Data is stored in the KQL database, not OneLake. To make a KQL table visible to other Fabric engines, you turn on OneLake availability, which creates a Delta copy of that table in OneLake. The platform maintains that copy, so this path isn't zero-copy the way a Lakehouse shortcut is but the result is similar: a table shortcut for that KQL table can be created in a Lakehouse.

This section covers only delta shortcuts for KQL tables and not KQL native shortcuts. KQL database can create its own shortcut to a OneLake delta source but this is a native KQL feature and not covered in this article.

Why I Use It

Keep ingestion in the KQL database but expose the same data in the Lakehouse, without building a second ingestion job.
Explore the table in the Lakehouse with Spark or T-SQL, the same way you explore any other Delta table.

Where It Shines

Analysts who work in Spark or T-SQL can read the time-series data without KQL database access, when the shortcut uses a delegated identity.
Power BI reports in Direct Lake mode can read the time-series data from the Lakehouse.

What to Keep In Mind

The Delta copy is a managed replica of the KQL table and there is always a delay, so this isn't a real-time view.
If OneLake availability is turned off and back on the replica is removed and rebuilt. During this period data is not available via a shortcut.
The access model is different from other sources because additional permissions are needed on the KQL database.

To learn more about OneLake availability for KQL check out this article.

When I Would Still Create Another Dataset

A shortcut is my preference, not a rule. I still create a separate dataset when the data is filtered, transformed and enriched and not an exact copy. Additional considerations may include performance optimization, data isolation and governance.

Final Thoughts

Table shortcuts in Fabric are extremely powerful and may enable many scenarios with unexpected data access paths. While it's important to manage your data redundancy and eliminate unnecessary copy operations the final architecture of your data estate may be much more complex than a simple choice of using a table shortcut.

I am curios, where in your Fabric estate are you duplicating data mainly to change tools, and what would make a shortcut-based access path a better choice?

Drop your thoughts and comments below.

Code review isn't enough for data pipelines

PratapIsHere-23 — Tue, 03 Nov 2026 14:33:03 GMT

Last year around April, there was a production issue in our data engineering system and we had to roll back a recently merged pull request. This pull request had minor code changes and was reviewed and approved by me and one more person on my team. The change was made in the code of one of the upstream tables of a pipeline, which resulted in a schema violation in a downstream table. The downstream table was part of a separate pipeline, hence it was missed by the data engineer who raised the pull request.

The change was quite small - and therefore tempting to approve quickly and be done with. Ideally, I should have looked at the pipeline runs in the development environment and verified all the downstream tables’ schema contracts and data. But practically speaking, this is a cumbersome and time-consuming task. Therefore, on busy days this usually gets skipped.

Post that incident, whenever I reviewed PRs, I tried to look at the changes at the data level as well - at least whenever time permitted. I took more time to review (which was of course a bit frustrating both for the PR author and the reviewer). This habit turned into a project over time. I have recently open-sourced it to help others in the data engineering community. This article covers that project.

Software teams have a safety net. Data teams mostly don’t.

In software engineering, a pull request triggers a build that runs a full suite of tests to ensure every component is working correctly and nothing is breaking due to the current set of changes. The build and the test results give an objective signal to the reviewer about the changes. It also helps newcomers to the team, who may not yet know all the components fully, to author their changes or review others' changes properly.

On the other hand, PRs raised in data engineering teams often have gaps in this process. A data engineer makes the code changes, creates/updates/deletes tables in the dev environment, tests the changes by running the pipelines and running other data quality tests. The first gap is here itself - governance. Unlike software engineering PRs, where there are coded tests for almost every related component that are run during the build, there are hardly any such tests set up to run automatically before merging any changes to production. There are usually no written schema contracts that the tests are run against. It depends on the practices followed in the team and the goodwill of the developer as to how thoroughly these tests are done.

The second major gap is lack of automation. The pull request itself does not trigger any build as such - one that tests all the code, pipeline runs, impact on downstream tables, etc. As I mentioned at the beginning of the article, the reviewer has to check the runs manually in the dev branch and, at times, run the tests to ensure nothing is breaking downstream.

The third gap is lack of visibility for new team members who do not have sufficient experience or knowledge about the team's data estate. They may miss out on testing some of the tables while making the changes or reviewing others' changes.

Most data teams have data quality checks in place to identify any issues in production and minimise the risk of corrupting data in any of the downstream tables. These are fine, but they catch the issues in production, which is the fourth and final gap - lack of proactiveness. Although we catch the issues in production through these checks, in most cases the damage is partly already done - be it delayed runtimes, failure to meet SLAs or simply wasted compute in rerunning some of the pipelines.

DEBIT - Data Engineering Build Impact Testing

In order to close the above gaps, I built DEBIT. On every pull request raised in our team, this framework runs the tests needed automatically at both the code level and the data level. It identifies the tables impacted, checks their contracts mentioned in YAML files, figures out what the downstream and upstream tables for these impacted tables are, and then runs the tests on the actual data in dev environments. Finally, it stores those results and displays them in the PR build log for the reviewer to inspect.

It targets Microsoft Fabric and lakehouse workflows, but the core is plain Python with no cloud dependency. The repo includes templates for GitHub Actions and Azure DevOps.

DEBIT is open source under the MIT license: github.com/pratap-is-here/debit

An Important Note

The framework does not replace any testing that the PR author must do. It supports it by bringing governance to the whole process of moving any changes from dev to prod, by ensuring increased thoroughness and efficiency as well as reducing the review workload on the reviewers. So the PR author is still strongly recommended to do the testing on appropriate data and, if needed, copy some of the data from prod to dev before testing.

DEBIT assumes the developer has already tested in dev against the latest prod data. If they haven’t, the validations fail, and it becomes visible in the PR.

That's deliberate. DEBIT checks that testing happened. It doesn't replace it.

2 Phases

Every PR runs through 2 phases.

Phase 1 is an offline hard gate. It validates the table declarations in the PR template, reads the git diff to find the affected tables, checks the registry for completeness, cross-checks the lineage for consistency, expands the scope along the lineage graph, and verifies that every table in scope has a complete contract. It needs no cloud access.

Phase 2 is optional and live. It builds a contract payload from the table registry, triggers a Fabric notebook through the Jobs API, and runs row count, freshness, schema and custom checks. It then reads the results back from OneLake and prints a structured pass/fail report. By default it warns instead of blocking, and you can switch it to a hard gate.

I made Phase 1 the hard gate because it’s deterministic and runs offline. The live checks depend on a Fabric run, so I wanted them to start as warnings until a team trusts them.

Lineage decides the Scope

Testing everything on every PR is slow, and testing only the changed table misses the damage downstream. So the PR author chooses how far validation reaches.

Each table lists its direct downstream consumers in a YAML file, and upstream is worked out from that at runtime. The author picks one of four modes in the PR description.

In the diagram, Silver F is the table the PR touched. Affected only validates just F. Upstream adds everything F reads from, six tables in all. Downstream adds what reads from F, three tables. Full lineage covers both directions, eight tables. Silver G and Gold J sit beside F in the graph, not above or below it, so they stay out of scope.

Metadata over Code

All the rules live in YAML, not in hardcoded logic.\

Five core files are part of the metadata:

subset_scope.yml: This file tells which tables, pipelines or notebooks are governed.
table_registry.yml: This file covers the per table contracts.
table_lineage.yml: This file mentions the downstream lineage.
notebook_outputs.yml and pipeline_outputs.yml: These files tell which tables each notebook and pipeline produces.

Two more handle execution freshness and custom check notebooks.

Here’s what a contract looks like, trimmed from the synthetic example in the repo:

silver.fact_orders: min_rows: 1000 freshness_hours: 24 freshness_column: order_date schema: required_columns: - order_id - customer_id - order_amount - order_date not_null_columns: - order_id - customer_id checks: - name: order_amount_non_negative sql: "SELECT COUNT(1) FROM {table} WHERE order_amount IS NOT NULL AND order_amount < 0" expect_eq: 0

There's no Python to write. Changing a threshold is a visible edit in a PR, with a name and a reason attached. A newer engineer can read the file to see what healthy means for a table.

Rolling it out

You don’t have to cover everything at once.

You can start with a set of pilot tables and adjust the checks and framework based on your team’s requirements. Once you are confident with the updates, you can expand to more tables by simply updating the metadata files. Assets that are outside the scope will remain untouched.

Did it help?

On our team, it cut PR review time by roughly 40% and production incidents by roughly 25 to 30%.

Treat those as our experience, not a benchmark. Your tables, your review habits and your starting point will differ.

What it doesn’t do

It assumes developers test in dev first. If the dev run is sloppy, DEBIT can only tell you that, not fix it.
A contract is only as good as what you write into it. Too tight and the checks raise false alarms, too loose and they miss real problems, so the numbers need upkeep.
The live checks only support Fabric today.
YAML can’t express everything, so there’s a custom SQL check as an escape hatch. I’d rather keep those few, or the files turn into code.
I built it for my own team, so expect rough edges.

Try it

You don't need cloud access to try it. The bundled example and the whole test suite run offline.

git clone https://github.com/pratap-is-here/debit cd debit pip install -r requirements.txt python examples/run_validation_example.py python -m pytest -q

The example walks one changed table through all four scope modes and prints a pass for each. The tests should report 114 passed.

DEBIT is open source under the MIT license: github.com/pratap-is-here/debit

If you review data PRs, I’d like to hear what’s missing. And if a field in the metadata files is awkward for your tables, open an issue with a small example of what you’d rather write. A real table that doesn’t fit is more useful to me than a star.

Views are my own.

Cross-Tenant Delegated OneLake Shortcut in Microsoft Fabric | End-to-End Tutorial

abiola_david — Mon, 02 Nov 2026 14:58:02 GMT

In this video, I walk through how to architect and implement a Cross-Tenant Delegated OneLake Shortcut in Microsoft Fabric. Discover how organisations can securely access and consume data stored in another Microsoft Fabric tenant without physically copying, moving, or replicating the data. Through a practical, step-by-step demonstration, you'll learn how to:

✅ Configure connectivity between two Microsoft Fabric tenants

✅ Implement delegated authentication for secure access

✅ Create a Cross-Tenant OneLake Shortcut

✅ Share data across organisational boundaries while maintaining governance and security

✅ Create Semantic model and Power BI report in the consumer tenant 🔍 What You'll Learn

✅ Understanding Cross-Tenant Delegated OneLake Shortcuts

✅ Microsoft Fabric multi-tenant data architecture

✅ Producer and Consumer tenant architecture design

✅ Configuring cross-tenant access in Microsoft Fabric

✅ Setting up delegated authentication using an Organisational Account or Service Principal

✅ Configuring workspace permissions and OneLake security

✅ Creating a Cross-Tenant OneLake Shortcut

✅ Accessing data across tenants without data duplication

✅ Security considerations and best practices for cross-tenant data sharing

🎯 Who Should Watch? This tutorial is ideal for:

🔹 Microsoft Fabric Data Engineers

🔹 Data Architects

🔹 Cloud Engineers

🔹 Analytics and Platform Teams

🔹 Anyone building secure multi-tenant data-sharing solutions

If you're learning Microsoft Fabric and want to understand how OneLake Shortcuts enable governed data sharing across organisational boundaries, this is a tutorial you don't want to miss. 💡 One of the biggest advantages of OneLake Shortcuts is the ability to eliminate unnecessary data duplication while maintaining security, governance, and a single source of truth across tenants.

🎥 Watch the video

https://www.youtube.com/watch?v=ihlWLRiia7Q&t=693s

and let me know your thoughts in the comments. 👍 If you find the content valuable, please Like, Share, and Subscribe for more practical tutorials on:

✅ Microsoft Fabric ✅ Azure Data Engineering

✅ Data Architecture

✅ Modern Data Platforms

#MicrosoftFabric #OneLake #OneLakeShortcut #CrossTenant #DelegatedAuthentication #MicrosoftFabricTutorial #DataEngineering #DataArchitecture #MicrosoftAzure #AzureDataEngineering #Lakehouse #DataSharing #ServicePrincipal #DataSecurity #CloudComputing #FabricLakehouse #DataPlatform #BigData #MicrosoftMVP #DatabricksMVP #AbiolaDavid

AI-Powered Fabric Optimization with Humans in Control

dimkalamaras — Tue, 27 Oct 2026 14:10:51 GMT

As Microsoft Fabric becomes central to business reporting and data engineering, inefficient workloads can affect more than execution time. They can consume shared capacity, compete with critical processes, and increase the effort required to keep analytics reliable and cost-effective. Identifying an optimization opportunity is valuable, but turning it into a trusted change requires evidence, clear ownership, and human judgment.

This solution brings together Azure SRE Agent, Microsoft Fabric, and GitHub to connect investigation with an explicit approval workflow. The agent analyzes workloads and proposes improvements, while a human reviewer decides whether those changes should proceed.

The central question is simple: How can we use AI to accelerate Fabric optimization without handing it unrestricted control over workloads or code?

The answer is a workflow in which the agent investigates, the human approves, and server-side functions validate and execute the permitted repository operations.

What This Solution Enables

The SRE Recommendation Approvals demo provides a reviewable path from a performance or capacity finding to a proposed Git change:

Evidence-based investigation using Fabric definitions, execution monitoring, and Fabric Unified Admin Monitoring (FUAM) data.
A dedicated approval inbox where reviewers can inspect the findings, rationale, estimated impact, and proposed changes.
An explicit human decision before a branch is created or approved edits are committed.
Traceability through the recommendation ID, reviewer identity, decision comment, branch, and resulting commit.
A deliberate stop before deployment, leaving merge, publication, and Fabric workspace synchronization outside the automated flow.

Approving a recommendation authorizes the proposed repository changes. It does not authorize an automatic production deployment.

Architecture Built Around Clear Responsibilities

The design separates analysis, human decisions, and repository writes rather than giving one agent responsibility for every action.

Azure SRE Agent: investigate and explain

The agent uses Fabric MCP, documented read-only APIs, and workload-specific skills to inspect the target workspace and item. Official Fabric skill documents are read through configured Code Access to the microsoft/skills-for-fabric repository.

FUAM adds consumption and capacity context through a read-only SQL endpoint connection. The agent must distinguish observations from likely causes and estimated benefits, check data freshness, and avoid presenting daily usage aggregates as exact measurements for an individual run.

Investigation does not run notebooks or pipelines, alter data, or change live Fabric settings. Managed identity provides Fabric API access; the agent does not receive the GitHub PAT used for repository writes.

A Fabric app: make the human decision visible

The approval experience is a Fabric SSO-protected app built with the Rayfin SDK. It presents a recommendation inbox and a detail panel containing the explanation, Markdown findings, proposed changes, and decision history.

Reviewers can add a comment and explicitly confirm Approve or Reject. Server-side decision Functions derive reviewer identity from the platform-issued caller context rather than trusting identity fields supplied by the browser.

The app's SQL database records the recommendation and its outcome. Its persisted decision states are intentionally simple: Pending, Approved, and Rejected.

Native Fabric UDFs: validate repository operations

Two Python User Data Functions support the agent workflow:

log_recommendation records the proposal, originating SRE thread, and discovered Git target.
commit_change_set checks the stored approval, exact recommendation branch, and permitted item paths before writing the change set.

The app's decision Functions create the branch on approval. The Python commit UDF subsequently writes the approved edits using its separately configured GitHub credential. PAT values remain server-side.

These checks do not replace code review: the agent must stay within the approved scope, and reviewers still assess the resulting diff before merge.

GitHub: keep changes reviewable and isolated

Each recommendation stores the investigated workspace's repository, base branch, and Git folder. The workflow never assumes that the workload belongs in the approvals app's own repository, and an existing approval is not silently retargeted when a workspace's Git connection changes.

Approved edits go to a dedicated recommendation branch. Pull request creation is off by default and requires an explicit reviewer request.

Azure, Fabric, and GitHub architecture diagram:

From Investigation to Approved Change

At a conceptual level, the workflow is:

Request an investigation. A user identifies the Fabric workload and analysis period in an SRE Agent conversation.
Collect evidence. The agent inspects definitions and monitoring, checks relevant FUAM data, and explains the findings and their limitations.
Record a recommendation when requested. The agent discovers the workspace's Git connection and invokes log_recommendation, preserving the real conversation thread and Git target.
Review the proposal. A human examines what will change, why it is recommended, and the expected impact in the approval app.
Approve or reject. Approval creates a branch from the stored base and sends the decision back to the originating SRE thread. Rejection notifies that thread without modifying GitHub.
Implement only the approved scope. Following approval, the agent invokes commit_change_set; the UDF validates the record and target before creating a non-force commit.
Report the outcome. Commit metadata is stored with the recommendation, and the agent reports the confirmed result in the original conversation. A PR URL is included only when a PR was explicitly requested and created.

The recommendation remains Approved after the commit. The commit SHA and optional PR URL provide implementation details without introducing a separate deployment status.

Αpproval and rejection workflow:

Why This Matters for the Business

The opportunity is not simply to generate optimization suggestions faster. It is to reduce the manual handoffs between identifying a problem, understanding its business relevance, assigning a decision, and preparing a reviewable fix.

For platform and engineering teams, this creates a consistent place to discuss performance and capacity trade-offs. For workload owners, it keeps the proposed scope and decision visible. For organizations adopting AI-assisted operations, it demonstrates a practical division of responsibility: AI supports analysis and implementation; people retain authority over change.

Any claimed improvement still needs validation. Reduced runtime or CU consumption does not automatically translate into lower invoices, and estimated benefits should be measured against an agreed baseline after the normal deployment process.

Implementation and Practical Scope

This article describes a demo reference implementation, not a production-hardened autonomous operations platform. Notebook optimization is the initial demonstration focus, with pipeline investigation and item-specific change handling providing a foundation for broader scenarios.

Important boundaries include:

Git targeting currently supports initialized connections to repositories hosted on github.com, not every Git provider or custom GitHub domain.
Live Fabric definitions can differ from repository content. A non-matching edit should be investigated, not used as permission to broaden the approved change.
Investigation permissions use a Contributor baseline in the demo; instructions constrain behavior but are not a substitute for technical access controls.
The UI prevents repeated submissions within the current page, but the workflow does not provide distributed locking or guaranteed exactly-once processing across concurrent requests.
The SQL record provides workflow traceability, not an immutable compliance audit trail.

Before production adoption, teams should assess least-privilege access, credential lifecycle, concurrency and recovery, audit requirements, and their existing release controls.

The source and setup guidance are available in the DimKal-Org/fabric-sre-approvals-demo repository, including:

The Fabric app, SQL entity, and decision Functions.
The native Python UDF implementation.
Agent instructions and companion skills for Fabric investigation, FUAM analysis, and the recommendation workflow.
Local tests, configuration guidance, and editable architecture diagrams.

Who This Pattern Is For

Fabric platform teams investigating reliability and shared-capacity pressure.
Data engineers who want actionable recommendations with explicit review boundaries.
SRE and FinOps practitioners connecting technical evidence with operational and consumption priorities.
Architects evaluating agentic operations without removing human ownership of change.

The goal is not to remove humans from Fabric operations. It is to give them better evidence, clearer choices, and a controlled path from insight to change.

Reassigning a Workspace in the Microsoft Fabric Admin Portal

abiola_david — Thu, 10 Sep 2026 13:38:41 GMT

Reassigning a Workspace in the Microsoft Fabric Admin Portal

When working with Microsoft Fabric, workspaces are an important part of organising and managing analytics content. A workspace can contain items such as Lakehouses, Warehouses, Notebooks, Data Pipelines, semantic models, and Power BI reports.

Depending on how a workspace is configured, it can be assigned to different types of capacity. There may be situations where a workspace that was originally running on a Power BI Pro shared environment needs to be moved to a Microsoft Fabric Capacity.

In this article, I'll show how to reassign a workspace to a Fabric Capacity using the Microsoft Fabric Admin Portal.

For this demonstration, I have a workspace named task wk, which is currently assigned to the Power BI Pro workspace type.

Current Workspace Configuration

I have already created the following workspace:

Workspace Name: task wk Workspace Type: Power BI Pro

At the moment, task wk is not assigned to my Fabric Capacity.

The objective is to move this same workspace from its current Power BI Pro setup and assign it to a Fabric Capacity.

Rather than creating a new workspace, I can simply reassign the existing workspace.

Open the Fabric Admin Portal

The first step is to open the Microsoft Fabric Admin Portal.

From the Fabric interface, open the settings menu and select Admin portal.

The Admin Portal provides administrators with tenant-level management capabilities, including the ability to manage workspaces and their capacity assignments.

For this task, I need to work with the Workspaces section.

Go to the Workspaces Tab

Inside the Fabric Admin Portal, select Workspaces.

This section provides a view of the workspaces available within the Fabric tenant.

I can search for my workspace by name rather than scrolling through the entire list.

In my case, I'm looking for:

task wk

Once I locate the workspace, I can see its current configuration.

The workspace is currently associated with the Power BI Pro workspace type.

Select Reassign Workspace

To change the capacity assignment, I select the ellipsis (...) next to task wk.

This opens a menu containing actions that can be performed on the workspace.

From the menu, I select:

Reassign workspace

This is the option I need to change the workspace's capacity assignment.

Select the Fabric Capacity

After selecting Reassign workspace, Fabric presents the available capacity options.

From here, I select the Fabric Capacity that I want to assign to the workspace.

The important thing is to make sure I select the correct capacity, particularly if the organisation has multiple Fabric capacities.

Once the appropriate Fabric Capacity has been selected, I confirm the reassignment.

Fabric then updates the workspace's capacity assignment.

Verifying the Workspace

After completing the reassignment, I can return to the Workspaces section in the Admin Portal and check task wk again.

The workspace should now show that it is assigned to the selected Fabric Capacity rather than its previous Power BI Pro setup.

The workspace itself has not been recreated. The existing workspace and its contents remain in place; what has changed is the capacity to which the workspace is assigned.

This is useful because I don't need to migrate all the workspace items into a new workspace simply because I want to change the capacity.

Why Reassign a Workspace?

There are several reasons why an organisation might want to reassign a workspace to Fabric Capacity.

One common reason is to make Fabric capabilities available to the workspace.

For example, an organisation may start with a traditional Power BI workspace and later adopt Microsoft Fabric. Moving the workspace to an appropriate Fabric Capacity can be part of that transition.

Capacity assignment can also be useful when an organisation wants to manage workloads using dedicated capacity rather than relying on shared capacity.

A Simple Before and After

In this example, the change can be summarised as:

Before task wk --> Power BI Pro

After the reassignment:

After task wk --> Fabric Capacity

The workspace name remains the same, and I don't have to create another workspace just to make the change.

Things to Consider

Before reassigning a workspace, it is worth checking that you have the appropriate administrative permissions and that the target Fabric Capacity is available.

It is also important to understand the implications of moving a workspace between capacity types, especially in an organisation where capacity usage, governance, and licensing are carefully managed.

The exact options available in the Admin Portal can also depend on the tenant configuration and the permissions of the administrator.

Switching Lakehouses in a Microsoft Fabric Notebook

abiola_david — Thu, 03 Sep 2026 14:06:26 GMT

When working with Microsoft Fabric Notebooks, it is common to work with data that lives in more than one Lakehouse. Instead of creating a separate Notebook for every Lakehouse, Microsoft Fabric allows you to attach multiple Lakehouses to the same Notebook and switch between them when needed.

In this article, I’ll demonstrate this using two Lakehouses: Sales and Marketing. The Sales Lakehouse contains a dbo.sales_analytics table, while the Marketing Lakehouse contains a dbo.marketing_analytics table.

Connecting Multiple Lakehouses to a Notebook

I started by creating a Fabric Notebook and attaching both the Sales and Marketing Lakehouses to it.

Once both Lakehouses are connected, they become available within the Notebook environment. This means I can work with data from either Lakehouse without having to create separate Notebooks.

For example, the Sales Lakehouse contains:

dbo.sales_analytics

while the Marketing Lakehouse contains:

dbo.marketing_analytics

The important part here is that both Lakehouses are connected to the same Notebook.

Switching Between Lakehouses

One useful feature of Fabric Notebooks is the ability to switch the active Lakehouse from within the Notebook interface.

Marketing Lakehouse is the current Lakehouse with pin indicator. If I intend to switch, I can select the Sales Lakehouse as the active Lakehouse and work with the sales_analytics table. To switch, right-click on the Sales Lakehouse (or click on the ellipsis). Then, I can click Select as default lakehouse

As seen below, the Sales Lakehouse has been promoted to the top of the Marketing Lakehouse.

I can also switch the active Sales Lakehouse back to Marketing and work with the marketing_analytics table, all within the same Notebook.

This can be particularly useful when building data engineering or analytics workflows where data is distributed across multiple Lakehouses.

Working with the Sales Lakehouse

After switching the active Lakehouse to Sales, I can access the sales data from the Notebook.

For example:

df = spark.sql("SELECT * FROM Sales.dbo.sales_analytics LIMIT 10")

display(df)

Because Sales is the active Lakehouse, the table can be referenced directly using its table name.

I don't need to create another Notebook just to work with the Sales Lakehouse.

Query Marketing Analytics Table

Note, I don't necessary need to switch to the Marketing Lakehouse before I can query the inherent marketing_analytics table!

For example, I can simply author the SparkSQL code shown before to read the data from the marketing_analytics

df = spark.sql("SELECT * FROM Marketing.dbo.marketing_analytics LIMIT 10")

display(df)

Being able to switch Lakehouses makes the Notebook much more flexible because I can move between different Lakehouse environments without leaving the Notebook or creating additional Notebooks.

Why Switching Lakehouses Is Useful

At first, this might seem like a small feature, but it can be very useful when designing Fabric data engineering solutions.

For example, imagine an organisation has separate Lakehouses for different business domains:

Sales Lakehouse └── dbo.sales_analytics Marketing Lakehouse └── dbo.marketing_analytics

A single Notebook can be connected to both Lakehouses. Depending on the task I'm performing, I can switch between them and work with the appropriate data.

This can help reduce the number of Notebooks required in a Fabric workspace and make development more convenient.

It can also be useful when demonstrating Fabric capabilities, developing data transformation logic, or working with multiple business-domain Lakehouses.

One Notebook, Multiple Lakehouses

The key takeaway is that a Fabric Notebook doesn't necessarily have to be tied to just one Lakehouse.

By connecting multiple Lakehouses and switching the active Lakehouse when required, I can use a single Notebook to work with data across different Lakehouse environments.

In this example, I started with the Sales Lakehouse, queried dbo.sales_analytics, switched to the Marketing Lakehouse, and then queried dbo.marketing_analytics — all from the same Fabric Notebook.

This provides a simple and convenient way to work across multiple Lakehouses while keeping related development activities within a single Notebook.

From ADF Inventory to a Fabric Operating Model: A Practical Migration Playbook (Part 2)

ssrithar — Mon, 24 Aug 2026 13:45:19 GMT

A practical guide to building a portable, metadata-driven ingestion framework for Microsoft Fabric. Learn how JSON configuration, Pipelines or Airflow orchestration, watermarks, retries, and audit tables work together to make data ingestion scalable and safe.

From ADF Inventory to a Fabric Operating Model: A Practical Migration Playbook

ssrithar — Tue, 11 Aug 2026 13:47:18 GMT

Migrating from Azure Data Factory to Microsoft Fabric is not a one-for-one conversion. This practical playbook helps you assess existing workloads, choose the right Fabric pattern—Mirroring, Copy jobs, Pipelines, or Notebooks—and validate the move safely through metadata-driven design, reconciliation, and phased cutover.

Understanding SHOWPLAN_ALL in Fabric SQL

abiola_david — Mon, 03 Aug 2026 19:00:00 GMT

As data engineers, we spend a significant amount of time writing SQL queries to ingest, transform, and analyze data. However, producing the correct result is only half the story. Equally important is understanding how the SQL Server Query Optimizer executes our queries.

One of the most effective ways to inspect the optimizer's decisions is by using SHOWPLAN_ALL.

In this article, I'll demonstrate how to use SHOWPLAN_ALL in SQL Server and Microsoft Fabric SQL Database, explain what it does, discuss a common pitfall, and show you how to resolve it.

What is SHOWPLAN_ALL?

SHOWPLAN_ALL is a session-level SQL Server setting that instructs the query optimizer to return the estimated execution plan instead of executing the query.

Rather than returning data, SQL Server provides detailed information about the physical operators it intends to use, allowing us to understand how the query will be processed before it runs.

This is particularly useful when:

Investigating slow-running queries.
Understanding optimizer decisions.
Identifying expensive operations such as sorts and scans.
Comparing different query implementations.
Tuning SQL for better performance.

Unlike PostgreSQL, MySQL, Oracle, or Databricks SQL, which support variations of the EXPLAIN command, SQL Server relies on SHOWPLAN_ALL and graphical execution plans.

Orders Table

For this walkthrough, I'll use anorders table in Fabric SQL

CREATE TABLE orders
(
    order_id INT PRIMARY KEY NOT NULL,
    order_date DATE NOT NULL,
    customer VARCHAR(20) NOT NULL,
    amount INT NOT NULL
);

After populating the table with data, I'll calculate a running total using a window function.

Sample Query

SELECT
    order_id,
    order_date,
    customer,
    amount,
    SUM(amount) OVER
    (
        ORDER BY order_date, order_id
    ) AS running_total
FROM orders
ORDER BY customer, order_date, order_id;

Without any execution plan settings enabled, SQL Server executes the query normally and returns the dataset.

Viewing the Estimated Execution Plan

To inspect how SQL Server intends to execute the query, enable SHOWPLAN_ALL.

SET SHOWPLAN_ALL ON;
GO

SELECT
    order_id,
    order_date,
    customer,
    amount,
    SUM(amount) OVER
    (
        ORDER BY order_date, order_id
    ) AS running_total
FROM orders
ORDER BY customer, order_date, order_id;
GO

Instead of returning rows from the orders table, SQL Server returns an estimated execution plan describing the physical operations that would be performed.

Although the exact operators depend on the optimizer and available indexes, the execution plan typically resembles the following sequence:

Read the data from the orders table.
Perform any required sorting for the window function.
Compute the running total using the Window Aggregate operator.
Apply the final ORDER BY.
Return the results.

This visibility into the optimizer's decision-making process is invaluable when diagnosing performance issues.

A Common Pitfall

One of the most common mistakes developers make is assuming that SHOWPLAN_ALL only affects the next query.

It doesn't.

SHOWPLAN_ALL is a session-level setting.

Once enabled, every subsequent query in the same session returns an execution plan instead of executing.

For example, after running:

SET SHOWPLAN_ALL ON;
GO

Even a simple query such as:

SELECT *
FROM orders;

returns the execution plan rather than the table data as seen below

If you're unaware that SHOWPLAN_ALL is still enabled, it can be quite confusing because every query appears to "stop working."

The Solution

The fix is straightforward.

Disable the session setting.

SET SHOWPLAN_ALL OFF;
GO

After turning it off, SQL Server immediately resumes normal execution.

Running the same query again returns the expected dataset.

Why This Happens

Many SQL Server settings persist for the duration of the current session.

SHOWPLAN_ALL is one of them.

Other commonly used session-level settings include:

SHOWPLAN_XML
STATISTICS IO
STATISTICS TIME
NOCOUNT

Understanding session scope is important when troubleshooting unexpected SQL Server behavior, particularly during performance tuning.

As data volumes continue to grow, query performance becomes increasingly important.

Execution plans provide insights that cannot be obtained simply by reading the SQL statement.

They help answer questions such as:

Is SQL Server performing a Table Scan or an Index Seek?
Is an unnecessary Sort operation occurring?
Which operator consumes the highest estimated cost?
Is the optimizer using a Window Aggregate efficiently?
Can the query be rewritten to reduce resource consumption?

These are exactly the questions that distinguish writing SQL from engineering performant SQL solutions.

Key Takeaways

If you regularly work with SQL Server or Microsoft Fabric SQL Database, keep the following in mind:

SHOWPLAN_ALL returns the estimated execution plan without executing the query.
It is a session-level setting, not a one-time command.
Every query continues returning execution plans until the setting is explicitly disabled.
Use SET SHOWPLAN_ALL OFF to restore normal query execution.
Learning to interpret execution plans is an essential performance tuning skill for data engineers and database professionals.

Final Thoughts

Window functions, Common Table Expressions (CTEs), and complex analytical queries are becoming increasingly common in modern data platforms. While writing these queries correctly is important, understanding how the SQL Server Query Optimizer executes them is what enables us to build scalable and efficient data solutions.

SHOWPLAN_ALL offers a simple yet powerful way to inspect the optimizer's strategy before a query is executed. Combined with graphical execution plans and tools such as STATISTICS IO and STATISTICS TIME, it forms an essential part of every data engineer's SQL performance tuning toolkit.

The next time you're optimizing a query, don't just verify that it returns the correct result—take a few minutes to examine how SQL Server plans to execute it. The insights you gain can often reveal opportunities for significant performance improvements.

Microsoft Fabric Just Made Quick Data Explorer Much Easier

Murtaza_Ghafoor — Mon, 03 Aug 2026 14:11:34 GMT

Tired of creating temporary notebooks just to troubleshoot a pipeline or check a table? Meet the Lakehouse Query Explorer—Microsoft Fabric’s new lightweight Spark SQL editor built for rapid exploratory work.

Choosing the Right Way to Run Python in Microsoft Fabric

apturlov — Tue, 21 Jul 2026 13:45:43 GMT

Fabric gives us several ways to run Python, and at first they can look overlapping.
In this post, I share the practical decision model I use to choose the right option based on execution mode, compute engine, and data access path. If you are code-first and want fewer wrong turns when moving from exploration to production, this guide is for you.

Breaking Barriers: Azure Databricks Unity Catalog Tables Can Now Be Stored Directly in OneLake

Srisakthi — Mon, 22 Jun 2026 14:09:32 GMT

Seamlessly Read & Write data to OneLake from Azure Databricks with Unity Catalog governance in place!

Playing the Zork game in Fabric as a Fabric App

svelde — Mon, 08 Jun 2026 13:39:19 GMT

Whether you are a pro-coder or a software maker, the new Fabric Apps feature offers an easy and powerful way to vibe code custom software applications and host them within Fabric.

Although Fabric is a very serious SaaS platform, I now have the tools to play the Zork game within Fabric!

I just deployed the Fabric App Hello World template and asked GitHub Copilot to merge the Visual Zorker into the Fabric App. Then, I only had to deploy it again. All in five minutes tops.

Let's check out how this is done.

Direct Lake Is Changing the Lakehouse vs Warehouse Debate in Microsoft Fabric

Tamanchu — Mon, 01 Jun 2026 13:55:33 GMT

Most Fabric discussions still focus on Lakehouse versus Warehouse.

I believe that's increasingly the wrong question.

Thanks to Direct Lake, many organizations can now go directly from Lakehouse to Power BI without introducing a Warehouse layer. But there are important trade-offs and hidden performance considerations that every Fabric architect should understand before making that choice.

Let's dive into what really drives the decision.

The Fabric Admin Trap: Scaling Your Cleanup

Pragati11 — Wed, 27 May 2026 13:41:05 GMT

We’ve all been there. It’s Friday afternoon, and you’re looking at your Microsoft Fabric tenant. It’s cluttered with dozens of abandoned test workspaces, half-finished projects, and “oops, I forgot to delete this” environments. You open the portal. You click. You wait for the page to refresh. You click again. You feel the rage slowly building. As admins, we are supposed to be power users, but we often spend more time navigating UI menus than actually managing our data. I decided enough was enough and turned to the Microsoft Fabric CLI (fab) to take back control. But the path to automation wasn’t a straight line.

A reference architecture for private API access from Microsoft Fabric

dimkalamaras — Fri, 22 May 2026 15:12:10 GMT

Calling private APIs from Microsoft Fabric — a reference architecture with Managed Private Endpoints and Azure Functions

Fabric Data Agents - The Orchestration Playbook for M365 or Teams experience

hasrikak — Wed, 20 May 2026 18:27:45 GMT

You’ve built a Fabric Data Agent - now where should users interact with it? This post compares the major orchestration paths across the native Fabric experience, Microsoft 365 Copilot, and Teams bot orchestration, with a practical lens on fit, identity, and user experience

Materialized Lake Views in Microsoft Fabric: Making Engineering Work Easier

Murtaza_Ghafoor — Mon, 18 May 2026 14:40:57 GMT

Managing data does not have to be painful. Learn how Materialized Lake Views in Microsoft Fabric combine simple SQL with automated updates to keep your data ready and your costs at lower levels.