Forum Discussion
Why the need for Dataflows and Datamarts when Semantic Models can do all of this already??
Hey OptalyticBI ,
you will not miss "best practices" when you stick to your working method. You extract, transform, and load data, then you create your model and fínally you visualize the data and share the report with your colleagues. In our organization there are many teams developing Power BI solutions targetting large audiences, there are many teams creating apps for their departments, or sections, we have ~2.4k Pro licences installed.
Regarding dataflows
We recommend everyone who is asking (not every one does, and no one has to) to use dataflows. The reasoning behind this is the different compute architecture, dataflows are performing way much faster then Power Query queries. With every compute second safed we safe money because we do not have to upgrade our capacities, at least not today.Regarding datamarts
Currently we do not use datamarts, and most probably we never will. With the advent of Microsoft Fabric there are other means to share "raw" data (not the semantic model and also not the data visualizations) with other teams or external guest users. A datamart or a SQL endpoint as we now call it with Microsoft Fabric makes data more accessible, data can be shared with a larger audience, because the lingua franca of data is SQL not python, and "unfortunately" not DAX.
From my perspective, you do not need to change your working method, but you have to keep an eye on
- resource consumption
- the need to share data assets between teams and/or external users
Hopefully, this helps to make up your mind.
Regards,
Tom
Like you, I was enjoying just using a semantic model as my source of truth, and with all users in our organisation having Pro licences, sharing data and reports was a doddle.
But, over time, as my semantic models grew a bit, I noticed that performance was tanking. That's when I discovered that when you reference tables in the semantic model (like if you aggregate or join/merge), it does not used any cached form of the data, but re-extracts everything from the database again as the source of the aggregation or join/merge. It's not very intuitive that it should do that, but it does explain the poor performance of semantic models that do that sort of thing.
The recommended alternative is data flows that need Premium Per User (PPU) licences, with an associated cost. Data flows extract the data and put it somewhere in the cloud in Power BI, and that can then be used by semantic models more quickly than from the database, although they will still re-extract from those sources multiple times when you do any aggregates or merges.
It doesn't look like there is any good way to short-circuit that either. As soon as you want t do something properly useful at a large enterprise level, the licence levels and associated costs expand.
😞
I haven't worked with data marts, so I am afraid I can't comment on those.
Regards,
K.