Forum Discussion

Txtcher's avatar
Txtcher
Helper V
1 year ago
Solved

Which if faster? Load into Data Model/Create Relationships, or Merge Queries

Our organization does not use Power BI to share reports. We are limited to using Excel. We have an API connector to Salesforce. The objects are huge, and I find if I create merges in Power Query, the...
  • v-aatheeque's avatar
    v-aatheeque
    1 year ago

    Hi Txtcher ,
    Thanks for reaching out to Microsoft Fabric Community forum.

    I understand you're looking to compare two approaches loading data into the Data Model and creating relationships vs. merging queries in Power Query based on performance in Excel.

    • When you load data into the Excel Data Model and establish relationships, you're leveraging PowerPivot, the engine behind the Data Model, to perform the heavy lifting. The VertiPaq engine, used by PowerPivot, compresses and optimizes data for DAX queries. By loading separate tables and creating relationships, you're using a more efficient columnar database structure, which is scalable for large datasets. This approach avoids loading everything into memory as a single flat table.
    • Once data is in the Data Model, you can use DAX to create calculated columns, measures, or aggregates based on the relationships, adding flexibility and enhancing analysis.
    • Merging large tables in Power Query can be slow, particularly with datasets from Salesforce. Since Power Query processes data in memory, large datasets can make the process time-consuming and resource-intensive. Merging is computationally intensive, and after the merge, the resulting large flat table can hinder further performance.
    • Loading into the Data Model and Creating Relationships
    • This approach works best when:Working with large datasets and complex relationships.
    • You need efficient aggregations or reporting.Combining multiple data sources.
    • Merging in Power Query,This method is ideal when:Handling small to medium datasets with simple relationships.Data transformation is required before reporting (e.g., generating a flat table)
    • You need to perform complex data transformations with intermediate results.
    • Loading data into the Data Model and using relationships is faster and more flexible, especially for large datasets. It minimizes upfront processing time and leverages Excel’s efficient in-memory analytics. Data Model and Relationships are more efficient for large datasets and complex reporting, while Merging in Power Query is better suited for smaller datasets and simpler needs, especially when a flat table is required.

    If this post helps then please consider Accept it as the solution to help the other members find it more quickly.
     
    if still you have problems on it please feel free to let us know !