Every day, I receive a report of the previous day's transactions (about 9,000 rows) that I append to my main Power BI report (stored in OneDrive).
Here is the setup in my Power BI:
- Load the email report attachments by combining them (if more than one).
- Clean the data.
- Append the cleaned data to my main report.
I believe this will load the report cumulatively as follows:
For instance, let's say we start on July 10:
- On July 10, the report from July 9 will be loaded, cleaned and appended...
- On July 11, the reports from July 9 and July 10 will be loaded, cleaned and appended...
- On July 12, the reports from July 9, 10, and 11 will be loaded, cleaned and appended...
And so on.
In the end, this repeated loading will consume a lot of memory, causing my reports to fail due to out-of-memory issues. Additionally, since Excel can hold a maximum of 1 million rows, and each report has 9,000 rows, my main report will crash after 111 days.
What is the best approach to rectify this issue?
Thanks.
