Forum Discussion

mh2587's avatar
mh2587
Icon for Super User rankSuper User
29 days ago
Solved

Microsoft Fabric Warehouse – Cross-Warehouse vs. Same-Warehouse Query Performance

Hi everyone,

I have a question regarding query performance and compute consumption in Microsoft Fabric Warehouse.

Suppose I have two tables:

  • FactSales in WarehouseA
  • DimProduct in WarehouseB

I join them using a cross-warehouse query:

SELECT * FROM WarehouseA.dbo.FactSales AS f
JOIN WarehouseB.dbo.DimProduct AS d
  ON f.ProductID = d.ProductID;

Now, suppose I copy both tables into WarehouseA and run the same query using tables from the same Warehouse.

My questions are:

  1. Would querying tables from the same Warehouse generally improve performance compared to cross-warehouse queries?
  2. Would it reduce Fabric capacity consumption (CU usage), or is the difference negligible in most cases?
  3. Does cross-warehouse querying introduce additional data movement or processing overhead?

I would appreciate any insights or real-world experiences. Thanks!

  • Hi mh2587​,

    Adding the "why" behind the observation above that there is no meaningful difference, plus a way to prove it on your own workload.

    Why it is usually the same. Warehouses in the same workspace are not separate servers. A cross-warehouse query runs on the same distributed SQL engine, which reads both tables' Delta/Parquet files directly from OneLake. There is no linked-server-style hop that moves FactSales or DimProduct between databases first:

    same-warehouse join  -> engine reads A.FactSales + A.DimProduct from OneLake
    cross-warehouse join -> engine reads A.FactSales + B.DimProduct from OneLake
                            (same engine, same read path, no copy step)

    So for questions 1 and 3: no inherent extra data movement, and similar performance. For question 2: CU is driven by data scanned and CPU used, which is the same in both layouts if the data is the same.

    What can make them differ in practice has to do with the copies rather than the location:

    statistics   -> copied tables have fresh stats, originals may not
    file layout  -> a fresh copy can be better compacted / V-Ordered
    data volume  -> a filtered copy scans less than the original
    caching      -> the second run of any test benefits from warm cache

    That is the usual explanation when someone sees the copied version run faster: it is the newer data layout, not the single warehouse.

    Measuring it on your data. Label both variants, run each a few times, and compare them in query insights:

    -- variant 1: cross-warehouse
    SELECT COUNT(*)
    FROM WarehouseA.dbo.FactSales AS f
    JOIN WarehouseB.dbo.DimProduct AS d ON f.ProductID = d.ProductID
    OPTION (LABEL = 'xwh_join');
    
    -- variant 2: both tables in WarehouseA
    SELECT COUNT(*)
    FROM WarehouseA.dbo.FactSales AS f
    JOIN WarehouseA.dbo.DimProduct_copy AS d ON f.ProductID = d.ProductID
    OPTION (LABEL = 'samewh_join');
    SELECT label,
           COUNT(*)                            AS runs,
           AVG(total_elapsed_time_ms)          AS avg_ms,
           AVG(allocated_cpu_time_ms)          AS avg_cpu_ms,
           AVG(data_scanned_remote_storage_mb) AS avg_remote_mb
    FROM queryinsights.exec_requests_history
    WHERE label IN ('xwh_join', 'samewh_join')
    GROUP BY label;

    Ignore the first (cold) run of each, and use COUNT(*) or an aggregate rather than SELECT * so client transfer time doesn't dominate. CPU time and data scanned are the closest proxies to CU.

    The one real limit: cross-database queries work across warehouses and lakehouse SQL endpoints in the same workspace. Across workspaces you need shortcuts, which is a design constraint more than a performance one.

    So my overall approach would be:

    keep shared dims in one warehouse, join across (no copies)
    -> keep stats and table maintenance current on both sides
    -> compare variants with labels in queryinsights if in doubt
    -> plan workspace boundaries around the same-workspace rule

    That gives you the governance benefit of shared dimensions without paying a performance penalty.

    AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.

3 Replies

  • AvdBerg's avatar
    AvdBerg
    Icon for Resolver III rankResolver III

    Hi mh2587​ 

    We are doing this exact thing, we have a few facts and dims in a department specific warehouse and for example a DimDate in a more generic warehouse the can be used by different departments. Our experiences so far:

    1. Would same-warehouse generally be faster than cross-warehouse?

    We have not found any significant differences in same-warehouse as cross-warehouse queries, we are a little early days so are monitoring it. But first results look good.

    2. Would it reduce CU consumption?

    Same anwer here, we have not found any differences in CU consumption so far.

    3. Does cross-warehouse querying add data movement or processing overhead?

    Depends on how you use this. for us it reduced the processing overhead as a hard requirement was separate warehouses for different departments, we did not want duplicate dims that are being used across departments.

    General warehouse                    Department warehouse
    
                          Server warehouse/
                           Semantic model

    Hopefully this helps you.

     

    Did I answer your question, please mark the post as a solution or consider a like.

  • jubinsoni's avatar
    jubinsoni
    Icon for Memorable Member rankMemorable Member

    Hi mh2587​,

    Adding the "why" behind the observation above that there is no meaningful difference, plus a way to prove it on your own workload.

    Why it is usually the same. Warehouses in the same workspace are not separate servers. A cross-warehouse query runs on the same distributed SQL engine, which reads both tables' Delta/Parquet files directly from OneLake. There is no linked-server-style hop that moves FactSales or DimProduct between databases first:

    same-warehouse join  -> engine reads A.FactSales + A.DimProduct from OneLake
    cross-warehouse join -> engine reads A.FactSales + B.DimProduct from OneLake
                            (same engine, same read path, no copy step)

    So for questions 1 and 3: no inherent extra data movement, and similar performance. For question 2: CU is driven by data scanned and CPU used, which is the same in both layouts if the data is the same.

    What can make them differ in practice has to do with the copies rather than the location:

    statistics   -> copied tables have fresh stats, originals may not
    file layout  -> a fresh copy can be better compacted / V-Ordered
    data volume  -> a filtered copy scans less than the original
    caching      -> the second run of any test benefits from warm cache

    That is the usual explanation when someone sees the copied version run faster: it is the newer data layout, not the single warehouse.

    Measuring it on your data. Label both variants, run each a few times, and compare them in query insights:

    -- variant 1: cross-warehouse
    SELECT COUNT(*)
    FROM WarehouseA.dbo.FactSales AS f
    JOIN WarehouseB.dbo.DimProduct AS d ON f.ProductID = d.ProductID
    OPTION (LABEL = 'xwh_join');
    
    -- variant 2: both tables in WarehouseA
    SELECT COUNT(*)
    FROM WarehouseA.dbo.FactSales AS f
    JOIN WarehouseA.dbo.DimProduct_copy AS d ON f.ProductID = d.ProductID
    OPTION (LABEL = 'samewh_join');
    SELECT label,
           COUNT(*)                            AS runs,
           AVG(total_elapsed_time_ms)          AS avg_ms,
           AVG(allocated_cpu_time_ms)          AS avg_cpu_ms,
           AVG(data_scanned_remote_storage_mb) AS avg_remote_mb
    FROM queryinsights.exec_requests_history
    WHERE label IN ('xwh_join', 'samewh_join')
    GROUP BY label;

    Ignore the first (cold) run of each, and use COUNT(*) or an aggregate rather than SELECT * so client transfer time doesn't dominate. CPU time and data scanned are the closest proxies to CU.

    The one real limit: cross-database queries work across warehouses and lakehouse SQL endpoints in the same workspace. Across workspaces you need shortcuts, which is a design constraint more than a performance one.

    So my overall approach would be:

    keep shared dims in one warehouse, join across (no copies)
    -> keep stats and table maintenance current on both sides
    -> compare variants with labels in queryinsights if in doubt
    -> plan workspace boundaries around the same-workspace rule

    That gives you the governance benefit of shared dimensions without paying a performance penalty.

    AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi mh2587​ 

    We would like to inquire whether have you got the chance to check the solutions provided by other users in community to resolve the issue. We hope the information provided helps to clear the query. Should you have any further queries, kindly feel free to contact the Microsoft Fabric community.