Forum Discussion
Model with loading very big CSV files - best practices.
Hi jaryszek,
Pre aggregating older data means keeping full detailed records for the recent period (like the last one or two years) but storing only monthly or yearly totals for older years. This makes your model much smaller and faster because old data rarely needs transaction-level detail. In Power BI, you can do this by splitting your fact table into recent and old data in Power Query, summarizing the old data by year/month and key fields, and then combining it back with the recent detailed data.
Thanks & Regards,
Prasanna Kumar
Hi jaryszek,
Thank you for reaching out to the Microsoft Fabric Forum Community.
Since your Power BI model loads over 50 GB of data from CSV files, you're facing performance issues in Power BI Desktop, especially with large visuals like matrix tables. To fix this, the best approach is to restructure your data into a star schema by separating it into smaller, related tables (like Date, Product, Region) instead of using one large flat table. You can also create summary tables in Power Query to reduce the data volume used in visuals. During development, load only a sample of the data using parameters to speed things up, while keeping the full data in the Power BI Service for publishing. Additionally, avoid overloading visuals with too much detail use drillthrough or summary views instead. If possible, consider moving your data to a proper database like Azure SQL or Synapse to enable advanced features like incremental refresh and DirectQuery for better scalability.
Tnaks & Regards,
Prasanna Kumar