Hello all,
Currentely we have the the problem that we cannot perform our full upload of Salesforce data into Power BI using a Phyton script.
The goal is that we load in the past 100 weeks of 2 backup files (opportunity.csv and account.csv), so a total of 200 files through a Phyton script that transforms this into a file where are saved 4 rows per week and a number of colums.
If we perform this action directely from Spyder (Phyton app) then it works out fine (this takes about 15 minutes). Only if we paste the same script in our PowerBI desktop (were we make the visuals from our data) and do the refresh it keeps on running endlessly and does not seem to complete. This is a new situation for us and we seem not te be able to figure out where it goes wrong, because a year ago we had no trouble at all with this process.
One thing that we know for sure is that our backup files increase in size over time. Currentely the files have a size of Account.csv 6.273 Kb and Opportunity.csv 10.514 Kb. If we look back into for instance december 2022 the files have a size of Account.csv 4.649 Kb and Opportunity.csv 7.027 Kb. We have the idea that the combination of the number of files and the size of the files create this problem.
What we have found as a "temporarely" solution is that we exclude some "big weeks" of files, but now for some time this influence al lot our outcomes in the visuals.
The current situation is that we can load in 14 files from the past weeks and if we try to load in more then the refresh does not finish. If we on the other hand include an extra 100 files from 2022 and earlier the refresh works fully whithout any problem.
Does anybody have any idea where this process goes wrong or gives us any direction where to go to be working towards a solution?
Thanks in advance for thinking with us,
Annette
