Data Warehouse topics

LakeHouse SQL Endpoint connection disruptions

BenT — Wed, 28 Oct 2026 15:18:29 GMT

Hello,

Since a couple of weeks, we are constantly experiencing the connection issue to the SQL Endpoint our Lakehouses whenever we try to run the Python code on a local computer.
we always have to try to establish the connection to the SQL Endpoint by running a query in Fabric or trying to connect to another on-premisses database, before being able to running any code (multiple times a day), but still the issue persist and we would have to do this multiple times for it to start working.

Are there any disruptions on the SQL Endpoint connections?

Regards,

Benjamin T

Fabric Warehouse as source in MS Access

JVDA98 — Tue, 27 Oct 2026 07:08:15 GMT

Hi All,

I have a client that is still using an Access application for some very specific modules. We recently migrated the on-prem database to Fabric & now some of the connections of that Access application needs to change to the Fabric Warehouse.

Using the ODBC driver, this seems to work, the only issue is that it looks like the user that is using the application has to have "VIEWER" access on the workspace, otherwise the connection cannot be successfully performed.

We've tried by removing the viewer access, and giving granualar connect + readData permission on the specific warehouse, but this throws a connection error (even before we can select a warehouse).
It seems like it needs to have "connect" permissions on the "server".

Does anyone have experience with this set-up? Best practices? Service principal,... doesn't seem an option at first sight.

Thanks !

Item Reference variable not saved on Lakehouse shortcut Target Item

arehman36 — Fri, 23 Oct 2026 16:43:13 GMT

Hello everyone,

I have a workspace with a Lakehouse called Test_LH. Test_LH contains a schema shortcut to a warehouse called TEST_WH which is in a different workspace. I have a library variable that contains an Item Reference to TEST_WH. When i press manage shortcut on my schema shortcut inside Test_LH, and then press 'Edit Target' -> 'Target Item', and then select my library varialbe that contains an item Reference to TEST_WH, the selection does not save. Now, if i press manage shortcut again, it will show that the Target Item is TEST_WH, rather than the library variable. I have pressed "Update all variables" but this has not fixed the issue.

Here are some things I have tried

Logging out of Fabric, closing all my tabs and logging in again.
Double checked to ensure that my item reference was correct

I am not sure why It will not show the Target Item to be my variable after selecting it.

Thanks,

Issue with the DL model deployement

Ar_Sh — Thu, 22 Oct 2026 06:19:38 GMT

I have a Direct Lake semantic model connected to a Fabric Warehouse. After deploying it from DEV to QA, the existing data source connection was not configured correctly, so I had to create a new connection using Workspace Identity.Is this expected behavior with Fabric deployment pipelines, or is there a way to preserve the target environment's connection settings?

erratic connection Power BI Desktop --> Fabric SQL DW

vanessa_fvg — Wed, 21 Oct 2026 14:50:04 GMT

We have an ongoing issue that when you connect to Fabric SQL DW via Power BI Desktop through imported mode ONLY SOMETIMES it gives this error below and other times you can connect easily. this is not for a big dataset it is literally 2000 rows.

Microsoft SQL: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)

I have changed the timeout values it makes no difference. When i look at the monitor in the SQL DW it doesn't even show that its querying. Not sure where this is getting stumped.

any ideas of where i can diagnose what is going on?

Trasaction deadlocked on lock resources

jnlucas — Wed, 21 Oct 2026 13:07:12 GMT

Hi,

I created a Dataflow Gen2 in Microsoft Fabric that loads multiple tables into a warehouse. Every time I run the dataflow, I receive the following error message:

Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Microsoft SQL: Transaction (Process ID 206) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction. Details: Reason = DataSource.Error;ErrorCode = 10478;DataSourceKind = Warehouse;DataSourcePath = Warehouse;DataSourceKind.2 = SQL;DataSourcePath.2 = qenzxdde3pdenc5pgrl56uss44-3nktrgkfckbedi5tclkxsvoogi.datawarehouse.fabric.microsoft.com;WH_Bronce;Message = Transaction (Process ID 206) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.;ErrorCode = -2146232060;Number = 1205;Class = 13;State = 51;ConnectionId = 090e3c20-db6d-492b-b2a6-48814400d26f;Microsoft.Data.Mashup.Error.Context = User GatewayObjectId: 714e5a6b-cca0-4a5a-bb6b-7c891e9991ba". Código de error: 104100. (Id. de solicitud: 4926601a-0aa6-4cdf-86d4-e84f3cbd1767).

Any idea what it might be?

Thank you in advance.

How to know Warehouse name on Query Window (SMSS)

alfBI — Wed, 14 Oct 2026 06:34:41 GMT

Hi,

I am using SSMS to connect Fabric WH. I have created server registrations to have friendly names, but even with that I have a problem when opening multiple query windows against different WH.

Every window is connected to a different WH but the only way to distinguish it is checking the wonderful String ID shown on the right bottom bar. Unfortunately the friendly name used to regisyter the server does not appear here.

Any idea about how to distinguish easily which WH corresponds to every opened query window?

Thx

NetSuite data in Fabric: where do you put KPI logic, Spark notebooks, Warehouse or DAX?

kiara-mehra35 — Mon, 12 Oct 2026 13:27:16 GMT

Hi all,

I’d like to ask how you’d structure KPI logic in a Fabric setup built on NetSuite data, because I keep going back and forth on where it should live.

The main source is NetSuite ERP, covering finance, sales, CRM, inventory and procurement, along with a few other enterprise systems. Fabric Data Pipelines handle the scheduled transfers into OneLake, and the Lakehouse is organized in Bronze, Silver and Gold layers to separate raw, refined and curated data. Fabric Notebooks with Spark do the cleansing, transformations, business rules and KPI calculations before anything reaches reporting. A Fabric Data Warehouse then serves structured datasets for cross-functional reporting, and Power BI semantic models with DAX measures and RLS sit on top, feeding 50+ dashboards across finance, sales, operations, inventory and executive reporting. Security runs through Entra ID, access controls, lineage and encryption.

The part I’m unsure about is KPI logic. Right now some of it is calculated in Spark before the data reaches the Warehouse, and the rest is defined as DAX measures in the semantic models. How do you decide what goes where? Do you push everything you can into the Gold layer and keep DAX thin, or keep the business definitions in the semantic model so they’re easier to change?

I’d also like to hear how you handle NetSuite ingestion. With pipelines running on a schedule, how do you deal with incremental loads, late-arriving changes and deletes? Do you ever do full reloads for certain tables?

On the Lakehouse vs. Warehouse split, what made you choose one over the other for the serving layer? Did you run into duplicated logic or extra copies of the data between them?

And with 50+ reports, do you keep one semantic model per domain or a few shared ones? How do you manage RLS in that setup, especially as self-service use grows and more people start building their own reports on top?

Has anyone gone through this and changed their approach later?

How are teams structuring Microsoft Fabric Warehouse for scalable ELT and reporting workloads?

codeautomation — Wed, 07 Oct 2026 21:13:48 GMT

I’m looking at architecture patterns for Microsoft Fabric Warehouse in environments where multiple data sources, transformation jobs, and Power BI workloads all depend on the same platform.

A typical flow might look like:

Source systems → ingestion → staging → transformations → Fabric Warehouse → semantic models → Power BI

I’m particularly interested in how teams are handling:

staging vs curated schemas
ELT transformations inside the Warehouse
incremental loading patterns
data quality checks before publishing
CI/CD across dev, test, and production
workload separation for engineering vs reporting
monitoring failed loads and long-running queries
deciding when to use Warehouse vs Lakehouse for the same solution

For teams running Fabric Warehouse at scale, what architecture patterns have worked best for keeping the platform maintainable and avoiding duplicated transformations?

Unable to Connect Azure Synapse Database to Power BI Desktop

ShivaPr — Wed, 07 Oct 2026 06:17:41 GMT

Hi Team,

While I am trying to connect Azure Synapse DB to Power BI Desktop getting error.

Steps performed

1.Obtained the Azure Synapse SQL endpoint and database credentials.
2.Opened Power BI Desktop and selected Azure Synapse Analytics as the data source.
3.Entered the Synapse SQL endpoint and database connection details.
4.Attempted to establish the connection.
5.Received an error indicating that Power BI Desktop is unable to establish a connection to the Azure Synapse database.

Can some one help me how to resolve it ?

DataflowsStagingLakehouse Size

nicolasf — Mon, 05 Oct 2026 15:25:42 GMT

Hello,

My DataflowsStagingLakehouse system storage is 103GB! I have many DF Gen2 with enabled staging, but I really don't understand why this lakehouse is so big!

How can I explain me that this storage is so big?

What are the good pratices for this kind of lakehouse ?

Near-real-time CDC into Fabric Warehouse: MERGE vs Mirroring vs Lakehouse?

jubinsoni — Sat, 03 Oct 2026 00:01:37 GMT

Hi all,

I need operational data from an Azure SQL Database available in Fabric for reporting with roughly 5 to 15 minutes of latency. There are about 30 tables, mostly updates, with some deletes. Business users will query it in T-SQL and Power BI.

I see three main approaches and would like to understand the trade-offs:

Mirroring the source database into Fabric, then building curated tables in a Warehouse on top of it
Incremental MERGE into Warehouse tables from a pipeline or copy job that reads changed rows (CDC or a watermark column)
Eventstream CDC → Lakehouse Delta tables, queried through the SQL analytics endpoint

Questions:

Which approach is recommended at this latency and scale, and where does each one break down?
How are deletes handled in each? Mirroring is the most direct, but what about MERGE-based loads?
How much does running MERGE frequently cost on a Warehouse (capacity usage, small-file or fragmentation effects)?
If you mirror, is it better to build curated tables in a Warehouse or use views over the mirrored database?

Thanks!

Concurrency in Fabric Warehouse: how can multiple pipelines write to the same table?

jubinsoni — Sat, 03 Oct 2026 00:00:24 GMT

Hi all,

I'm designing ingestion for a Fabric Warehouse where several pipelines could write to the same tables. Examples include an hourly incremental load running at the same time as a backfill, and a few source systems landing in a shared fact table.

My understanding is that Fabric Warehouse uses snapshot isolation, and that two transactions modifying the same table at the same time can conflict, with one failing. I'd like to design around that from the start instead of finding out in production.

Questions:

Is conflict detection at the table level, or finer-grained? Does it behave differently for INSERT-only loads vs UPDATE, DELETE or MERGE?
What patterns are people using to avoid conflicts? Options I'm considering:
- Serializing writes per table through pipeline dependencies or a control table
- Landing each source in its own staging table, then running one consolidating MERGE
- Retry logic in pipelines for conflict errors
Do long-running read queries (for example, Power BI refreshes) block or affect writes?
Any monitoring tips? For example, which DMVs or Query Insights views show conflicts or long transactions?

Thanks!

Varchar(MAX) in SQL Endpoint

fabanalytics — Tue, 29 Sep 2026 15:00:09 GMT

I'm testing VARCHAR(MAX) support in a Fabric Lakehouse SQL Analytics Endpoint and discovered that it only worked after:

1. Enabling New Metadata Sync (Preview)
2. Creating a new Lakehouse after enabling the setting

Existing Lakehouses didn't seem to pick up the capability.
Is this officially expected, or is there a migration/upgrade path for existing SQL Analytics Endpoints?

How are teams using AI agents to automate modern data warehouse workflows?

codeautomation — Mon, 28 Sep 2026 20:23:50 GMT

I am exploring how AI agents can help improve data warehouse operations by automating repetitive tasks and assisting data teams with faster decision-making.

Some areas I am interested in:

Automating data pipeline monitoring and issue detection
AI-assisted SQL generation and optimization
Automatically identifying data quality issues
Triggering workflows based on business events or data changes
Generating documentation and insights from warehouse metadata

With platforms like Microsoft Fabric Data Warehouse, how are teams approaching AI integration?

Are you using:

Fabric pipelines with AI-powered automation?
Copilot or LLM-based assistants for warehouse development?
Custom AI agents connected with data warehouse APIs?

Would love to hear real-world architecture patterns and best practices from data engineers working with Fabric.

Data Warehouse deployment fails when new NOT NULL column added in schema

anon97242 — Thu, 24 Sep 2026 16:31:01 GMT

Greetings!

We have a deployment pipeline that we are attempting to move a warehouse schema change through, we are adding some not null columns.

The deployment from our Dev environment to Test is failing with the following error:

Microsoft SqlClient Data Provider: Msg 24735, Level 16, State 1, Line 1 Only nullable columns can be added to an existing table. SqlMSBuild: Script execution error. The executed script: ALTER TABLE [DM_Schema].[SummaryTable] ADD [NewColumn] INT NOT NULL,CONSTRAINT [SD_SummaryTable_2d66d69a5ee8421398ab13068f0c28b4] DEFAULT 0 FOR [NewColumn]

A couple of questions:
1) Why is this blocking? I can't seem to locate where NOT NULL columns are not supported by deployment pipeline? Is this a legitimate bug?

2) Why is does it appear to (rightly) add a DEFAULT 0 constraint even though no such constraint was defined in DEV environment?

3) Our current process with complicated schema changes is to use Schema Compare tool in VS Code, and promote complicated schema changes outside of the deployment pipeline, adding a column generally is not considered a complicated schema change, so just wondering if there is a way to do this within deployment pipeline?

Thanks!

OneLake RLS Fails Through Consumer schema Shortcut to other workspace

MahdiG — Wed, 23 Sep 2026 01:56:14 GMT

I’m testing a Microsoft Fabric producer/consumer architecture using OneLake Security RLS and a schema shortcut.

The producer workspace contains an IT Lakehouse with three shared tables: two protected by RLS and one without RLS. The IT Lakehouse item is also shared directly with the LOB user.

When the user accesses the producer Lakehouse directly from the OneLake catalog, the RLS works correctly.

However, when the same tables are accessed through the LOB Lakehouse shortcut using the consumer SQL analytics endpoint, only the table without RLS works.

The two RLS-protected tables fail with this error:

Unable to sync the table due to insufficient permissions.

Error code: TableAccessForbidden

The main question is whether this is an expected limitation of OneLake Security RLS over shortcuts with the consumer SQL endpoint, or whether additional permissions or configuration are required.

Views in Warehouse Breaks Source Control Sync

kozman — Tue, 22 Sep 2026 23:02:54 GMT

When I create a view in a warehouse and have it point to a table in a corresponding lakehouse in the same workspace, I am unable to sync a new workspace from a git branch. Reason is that the lakehouse is also new and tables/fields have not been established yet. This causes the dependency check to fail a git update. Adding a selective sync feature to Fabric will help to alleviate this problem as I would be able to do an update in two passes. Alternatively, having the ability to turn off dependency checking when updating from git would work as well. But currently, using views in warehouses that point to anything outside of that warehouse is absolutely broken from a CICD perspective.

Has anyone run into this problem and found a solve for it? According to Microsoft docs, views can only point to lakehouses in the same workspace, otherwise I would have put these warehouses in a different workspace and mitigate the issue that way.

Unable to Delete Warehouse

ARatcliffeTP — Tue, 22 Sep 2026 12:39:21 GMT

I was following along with a Lab: Explore data warehousing in Microsoft Fabric (Lab 06). I started the optional part involving a Semantic model but cancelled when I couldn't seem to select "My workspace" which is where I was working. However, I now cannot delete the warehouse within the workspace.

Error: This warehouse can't be deleted because there are warehouses that were cloned from it. Select View item lineage to see them.

I'm new to Fabric and so tried to use Copilot to solve the issue. Ultimately, it has suggested I post here: "From what you've described, it sounds like an orphaned transient clone reference left behind by the optional semantic model exercise, rather than a real warehouse you can delete yourself. The Fabric team or support would need to clear the metadata."

My workspace contains no other items. System storage is empty. Warehouse has been emptied. sys.databases returns only master and TempWarehouse.

Preparing Enterprise Data Warehouses for AI-Powered Applications

codeautomation — Sat, 19 Sep 2026 07:35:23 GMT

Hi everyone,

I am exploring how organizations are preparing their enterprise data platforms for AI-powered applications.

As more teams start building AI assistants and intelligent applications, having clean, structured, and accessible data becomes increasingly important.

I would like to understand how the community is approaching this with Microsoft Fabric Data Warehouse.

Some questions:

- What data modeling approaches work best when preparing warehouse data for AI and analytics workloads?

- How are teams balancing traditional BI reporting requirements with new AI use cases?

- What strategies are you using for maintaining data quality and governance at scale?

- Are there recommended patterns for connecting AI applications with enterprise warehouse data securely?

Would love to hear practical experiences and lessons learned from teams working with Fabric Data Warehouse.

Thanks!