Forum Discussion
MS Fabric : Manage Aggregations Support
Hi Tcs2022 ,
Thanks for reaching out to Microsoft Fabric Community.
Thanks to cengizhanarslan and Murtaza_Ghafoor for the detailed explanations.
Fabric can handle very large data volumes at the storage and compute layer. But, Direct Lake semantic models do not currently support classic Manage Aggregations, and fallback to DirectQuery is expected once table limits are exceeded, even for higher-grain queries.
If aggregation awareness across mixed grains is a hard requirement, a traditional Power BI composite model using Import for aggregates and DirectQuery for detail remains the supported approach.
Regarding Dataflows, they are intended for reusable data preparation and incremental refresh scenarios, and very large initial ingestions are constrained by refresh and capacity limits.
For ingestion at this scale, Fabric scenarios typically rely on Lakehouse ingestion patterns such as pipelines or notebooks, with aggregates materialized separately.
Hope this helps. Please reach out for further assistance.
Thank you.
Thanks for the prompt reply. I did consider splitting into multiple facts,however the total rows 100 B /3 B capacity would mean splitting into 33 facts which is not feasible.Business expects reporting at higher & lower grain .hence only agggregtes will not suffice.
Aggregate awareness is aMUST for any BI tool . It is a surprise that MS Fabric doesnt support Manage aggregtions. In the absence of manage aggregtion in msfabric, does it mean, ms fabric is not suitable for reporting large data bolume beyond the capacity limit as fallback will always query in direct mode even if the user creates adhoc report t higher grain.
Can we consider not using msfabric but create a composite model in pbi wherein data is imported at higher grain and lower grain is sourced from one lake thereby implement manage aggregation in the traditional pbi
Can we consider creating datflow to move daily incremental load of 95 M rows to dataflow & implement manage aggregtion.we are avoiding direct query to snowflake(our storage) as it incurs processing cost. However we encountered error during our initial load of 100+B rows. Is there a solution to load 100+ B rows in dataflow as one time load & then incremental load of 95 M per day.
Murtaza_GhafoorSuper User
Use Case:
The organization needs to deliver BI solution on a 100B+ row dataset with both high-level summaries and transaction-level drill-downs, while avoiding costly DirectQuery to Snowflake.
Proposed Solution:
Enterprise-grade Fabric architecture uses spark notebooks or pipelines to ingest data into a Delta Lakehouse with partitioned, incremental loads. High-value aggregation tables (daily, monthly, and business rollups) are stored as Delta tables and loaded in Import mode, while detailed data is accessed via Direct Lake in a composite semantic model with aggregation awareness and drill-through enabled. This approach avoids Snowflake DirectQuery, leverages OneLake and Fabric compute, and ensures predictable, controlled capacity costs.If this helped, ✓ Mark as Solution | Kudos appreciated