Forum Discussion
Model with loading very big CSV files - best practices.
Hi jaryszek,
Pre aggregating older data means keeping full detailed records for the recent period (like the last one or two years) but storing only monthly or yearly totals for older years. This makes your model much smaller and faster because old data rarely needs transaction-level detail. In Power BI, you can do this by splitting your fact table into recent and old data in Power Query, summarizing the old data by year/month and key fields, and then combining it back with the recent detailed data.
Thanks & Regards,
Prasanna Kumar
Hi jaryszek,
Yes, normalizing the fact table and replacing text with integer surrogate keys will make it faster but the real gain comes from reducing cardinality, removing unused columns, and pre aggregating older data.
Thnaks & Regards,
Prasanna kumar
What do you mean by
pre aggregating older data.?
It means that for older years you are using aggregated data and for the newest you are using granularity like days? How to implement it?