Forum Discussion
Star Schema From Flat Files - Performance
- 4 years ago
Anonymous None of this has to do with performance. Only reusability. The way dataflows work is that dataflows transform the data from the source systems and store the transformed data in Azure Gen2 blob storage. Power BI Desktop files then use import mode datasets to import the transformed data from the blob storage. So, some performance benefits potentially for the desktop file in that it isn't doing the transformation steps itself. However, the big wins with dataflows are that the source system is only bothered once by the dataflow and not by a bunch of individual PBIX files all importing the same data. In addition, the dataflow can be used by lots of reports but you only have to do the transformation steps once in the dataflow. A dataflow is simply a reusable query. It uses Power Query, the same technology used in Power Query Editor.
Live datasets are a way to take all of this a step further and make datasets essentially resusable by a large number of PBIX files. This live dataset could use its own Power Query queries or dataflows to import the data. In either case, the source system is only being bothered once for refreshes.
All of this said, if you aren't planning to reuse the datasets/queries then you might as well just create the queries in the PBIX itself. Once you publish to the Service, refreshes will be done in the Service just like dataflows so no difference in performance, both are being executed in the service using the exact same technology (Power Query). Now, some will argue that best practice is to use dataflows regardless because they can be reused even if you currently have no plans to reuse them. Yes and no. If you use dataflows, remember that you now have an additional import step so to get the latest data, first your dataflow has to refresh and then your dataset needs to refresh from that refreshed dataflow data. So this can add a delay potentially in the velocity of your data refresh process.
- 4 years ago
Anonymous Sure. My opinion on that subject follows your reasoning and I have gone back and forth between columns vs. measures several times. The answer is that it depends. However, if you have no need for dynamic interactivity in your calculations then I tend to believe that columns are better. One, the report is faster as you don't have complex calculations going on in measures that are executed when visuals are displayed just base aggregations of columns. Second, columns tend to be safer, especially for self-service BI. With measures you can't control the context in which users use the measures so you can end up with "incorrect" results. This is especially true with measure totals as measure totals need to be constructed with a specific context in mind. So, I'm with you on the 3rd item, in all likelihood, columns are the way to go in your case.
Anonymous Dataflows are good. If you you use Power Query in the Desktop. Create a single query that imports the file. Right-click and choose "Disable load". Now, right-click again and choose Reference. Now your query will be loaded and cached once and used by the other queries that reference it without reloading all of the data for every individual query.
- Anonymous4 years agoNot applicable
That is super smart! Thank you Greg_Deckler. If I can, should I do this in Dataflows (vs PBI Desktop)? Would it improve the performance or it's not really that big change?
- Greg_Deckler4 years agoCommunity Champion
Anonymous Dataflows are really about reusability versus performance. So, if you expect to create multiple desktop PBIX files using the same data, then go with dataflows. However, you might be better off creating a "golden dataset" using straight Power Query queries (non-dataflows) and publishing just the dataset. Then, connect live to that dataset from other PBIX files.