Forum Discussion
Star Schema From Flat Files - Performance
- 4 years ago
Anonymous None of this has to do with performance. Only reusability. The way dataflows work is that dataflows transform the data from the source systems and store the transformed data in Azure Gen2 blob storage. Power BI Desktop files then use import mode datasets to import the transformed data from the blob storage. So, some performance benefits potentially for the desktop file in that it isn't doing the transformation steps itself. However, the big wins with dataflows are that the source system is only bothered once by the dataflow and not by a bunch of individual PBIX files all importing the same data. In addition, the dataflow can be used by lots of reports but you only have to do the transformation steps once in the dataflow. A dataflow is simply a reusable query. It uses Power Query, the same technology used in Power Query Editor.
Live datasets are a way to take all of this a step further and make datasets essentially resusable by a large number of PBIX files. This live dataset could use its own Power Query queries or dataflows to import the data. In either case, the source system is only being bothered once for refreshes.
All of this said, if you aren't planning to reuse the datasets/queries then you might as well just create the queries in the PBIX itself. Once you publish to the Service, refreshes will be done in the Service just like dataflows so no difference in performance, both are being executed in the service using the exact same technology (Power Query). Now, some will argue that best practice is to use dataflows regardless because they can be reused even if you currently have no plans to reuse them. Yes and no. If you use dataflows, remember that you now have an additional import step so to get the latest data, first your dataflow has to refresh and then your dataset needs to refresh from that refreshed dataflow data. So this can add a delay potentially in the velocity of your data refresh process.
- 4 years ago
Anonymous Sure. My opinion on that subject follows your reasoning and I have gone back and forth between columns vs. measures several times. The answer is that it depends. However, if you have no need for dynamic interactivity in your calculations then I tend to believe that columns are better. One, the report is faster as you don't have complex calculations going on in measures that are executed when visuals are displayed just base aggregations of columns. Second, columns tend to be safer, especially for self-service BI. With measures you can't control the context in which users use the measures so you can end up with "incorrect" results. This is especially true with measure totals as measure totals need to be constructed with a specific context in mind. So, I'm with you on the 3rd item, in all likelihood, columns are the way to go in your case.
These are great tips Greg_Deckler - thanks again!
Sorry for being a pain, but could you help me to understand why publishing a single data set (and connecting Power BI to it) is a better option vs dataflow? Isn't it the same from the performance perspective? I may be ignorant, but to me it seems that both queries and transformations will be executed in the cloud - am I right?
To clarify, I am not planning to reuse these datasets/dataflows for other reports.
- Greg_Deckler4 years agoCommunity Champion
Anonymous None of this has to do with performance. Only reusability. The way dataflows work is that dataflows transform the data from the source systems and store the transformed data in Azure Gen2 blob storage. Power BI Desktop files then use import mode datasets to import the transformed data from the blob storage. So, some performance benefits potentially for the desktop file in that it isn't doing the transformation steps itself. However, the big wins with dataflows are that the source system is only bothered once by the dataflow and not by a bunch of individual PBIX files all importing the same data. In addition, the dataflow can be used by lots of reports but you only have to do the transformation steps once in the dataflow. A dataflow is simply a reusable query. It uses Power Query, the same technology used in Power Query Editor.
Live datasets are a way to take all of this a step further and make datasets essentially resusable by a large number of PBIX files. This live dataset could use its own Power Query queries or dataflows to import the data. In either case, the source system is only being bothered once for refreshes.
All of this said, if you aren't planning to reuse the datasets/queries then you might as well just create the queries in the PBIX itself. Once you publish to the Service, refreshes will be done in the Service just like dataflows so no difference in performance, both are being executed in the service using the exact same technology (Power Query). Now, some will argue that best practice is to use dataflows regardless because they can be reused even if you currently have no plans to reuse them. Yes and no. If you use dataflows, remember that you now have an additional import step so to get the latest data, first your dataflow has to refresh and then your dataset needs to refresh from that refreshed dataflow data. So this can add a delay potentially in the velocity of your data refresh process.