The ultimate Microsoft Fabric, Power BI, Azure AI, and SQL learning event: Join us in Stockholm, September 24-27, 2024.
Save €200 with code MSCUST on top of early bird pricing!
Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started
** Queries on Formula & Storage Engine for Power BI Experts **
Excerpt from the 2 articles that led to confusion:
EnterpriseDNA article says:
"Simple queries (SUM, MIN, MAX) can be answered by the Storage engine, otherwise a datacache is sent to Formula Engine for it to compute an answer."
It adds later that "Storage Engine, being faster, handles simple queries and can do it all by itself"
SQLBI article says:
"The queries sent to the storage engine might vary from a simple retrieval of the raw table data to more complex queries aggregating data and joining tables. The storage engine returns data in an uncompressed format, regardless of the original format of the data"
Questions
1) Does Storage Engine (SE) answer simple queries (EnterpriseDNA) or provides raw data (SQLBI) for Formula engine (FE) to compute an answer?
2) If SE is capable to answer few simple requests, is FE then relegating itself when it finds the optimal Query Plan and delegates Storage Engine to compute?
2) For such simple queries, by saying "SE can do it all by itself", is EnterpriseDNA implying that SE does not need FE and can send the answer directly to Summary Table for Power BI visual or are they indicating that SE can process queries using its in-memory data without relying on its compressed data?
I asked the same question on my LinkedIn Profile- would update if I find an answer there
Solved! Go to Solution.
Both EnterpriseDNA and SQLBI agree that Storage Engine (SE) deals with simple queries, SE is faster and use multiples cores, it has a limited/basic functions as mentioned but it can filter, join and summarize very well. EnterpriseDNA says SE can handle them alone, while SQLBI adds that SE deals with different types of queries and gives the data to Formula Engine (FE) for further work. FE can handle sophesticated and complex calculations over the data cache (raw data) retuned by SE, it's singles threaded (slower).
Yes, when FE figures out the best way to handle a query, it can let Storage Engine (SE) do the actual work because SE is good at it.
When EnterpriseDNA says "SE can do it all by itself," it means SE can handle simple queries alone without relying too much on Formula Engine (FE), such as when there are no callbacks. It doesn't skip FE but shows SE can manage simpler tasks independently, whether it's using in-memory or compressed data. SE returned non compressed data into data cache from the vertipacq engine.
⭕ Subscribe and learn Power BI from these videos
⚪ Website ⚪ LinkedIn ⚪ PBI User Group
additionally a good way to understand what the storage engine does and what the formula engine does is it to play with it in dax studio and run queries to get a feel for it, its a lot of information. Its one of my favourite things to spend hours on 😅
effectively the formula engine and storage engine(s) work together, but they each have their own 'jobs'.
Proud to be a Super User!
Both EnterpriseDNA and SQLBI agree that Storage Engine (SE) deals with simple queries, SE is faster and use multiples cores, it has a limited/basic functions as mentioned but it can filter, join and summarize very well. EnterpriseDNA says SE can handle them alone, while SQLBI adds that SE deals with different types of queries and gives the data to Formula Engine (FE) for further work. FE can handle sophesticated and complex calculations over the data cache (raw data) retuned by SE, it's singles threaded (slower).
Yes, when FE figures out the best way to handle a query, it can let Storage Engine (SE) do the actual work because SE is good at it.
When EnterpriseDNA says "SE can do it all by itself," it means SE can handle simple queries alone without relying too much on Formula Engine (FE), such as when there are no callbacks. It doesn't skip FE but shows SE can manage simpler tasks independently, whether it's using in-memory or compressed data. SE returned non compressed data into data cache from the vertipacq engine.
⭕ Subscribe and learn Power BI from these videos
⚪ Website ⚪ LinkedIn ⚪ PBI User Group
Thanks @Fowmy
Your answer helps with Q2 & Q3
I'll see if someone else responds to Q1- or else marks yours as an answer
From your overall comment, for Q1, I feel the word "Answer" is an inapt word by EnterpriseDNA, it supplies the answer or datacache back to FE- so I will assume SQLBI is using the right terminology there
@GulianiG
To answer Q1, The formula engine turns a DAX query into a query plan with steps for execution. Each step corresponds to an operation by the formula engine. Storage Engine (SE) fetches data from the Tabular model and sends it to FE for further calculations. When you asked about "SE answering simple queries,", yes it true, it means SE handles straightforward queries directly for visualization without needing FE for additional calculations. For the second part about "SE providing raw data," it's always true as FE can't access data; SE retrieves and supplies data to FE, regardless of the query complexity.
So, there two statements in the question, both are true.
⭕ Subscribe and learn Power BI from these videos
⚪ Website ⚪ LinkedIn ⚪ PBI User Group
Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.
Check out the August 2024 Power BI update to learn about new features.
User | Count |
---|---|
108 | |
77 | |
73 | |
47 | |
39 |
User | Count |
---|---|
137 | |
108 | |
69 | |
64 | |
56 |