Forum Discussion
Best Practice for Large Pre-Aggregated Fact Table: Import vs DirectQuery (Databricks, Power BI Embed
- 11 months ago
Hi nitinbourai
For very large, pre-aggregated datasets like yours—500 million to 1 billion rows—using Import mode in Power BI is generally preferred, especially since your metrics are already calculated and you’re avoiding heavy DAX. Import mode provides the best performance for slicing, filtering, and visual interactions because the data is cached in-memory, but it does require sufficient capacity to load and refresh such a large volume. With an A6 Power BI Embedded capacity, you may be able to handle this dataset, but you need to carefully monitor memory usage and refresh times, as very large imports can approach or exceed the capacity limits. DirectQuery can reduce memory pressure because data stays in Databricks, but it generates many live queries during interactions, which can slow down reports and significantly increase compute costs on Databricks. In practice, for pre-aggregated datasets where performance is critical, Import mode is typically the best choice, possibly combined with incremental refresh to manage refresh duration and memory usage. DirectQuery is more suitable if your dataset cannot fit in memory or if real-time data is required, but you should carefully test query performance and Databricks cost implications before committing.
Hi nitinbourai ,
Thank you for reaching out to Microsoft Fabric Community.
Thank you Poojara_D12 for the prompt response.
I wanted to check if you had the opportunity to review the information provided and resolve the issue..?Please let us know if you need any further assistance.We are happy to help.
Thank you.
- nitinbourai11 months agoNew Member
On high level , it seems the import mode is the best solution. Need to work on the recommendations shared.
No help required as of now , will reconnect if I have anything else.