Forum Discussion
Save downloaded API data to minimize API calls
- 1 year ago
I have a hammer and everything looks like a nail. And my hammer is called Excel.
With excel I would folow these steps:- create a Power Query "API Data" to get the API data
- Load the result into an excel table which creates a table named API_Data
- Create a query "Collected Data" on the excel table API_Data
- And now the tricky bit:
- Load the "Collected Data" query ALSO to a table in Excel to get the excel table Collected_Data
- Change the Colledted_data to use the table Collected_Data
- Yes! It is self referencing
- Remove the excel table API Data.
- Finaly Change "Collected Data" query to combine it with "API Data"
- Change "API Data" to only return data that does not alredy exist in "Collected Data"
- Change "Collected Data" to do an "Append Queries" of "API Data"
You may think this reults in a circular reference, but it doesn't. This is because the Excel Table is NOT the Power Query...
Have fun! I don't know if you want to automate the refresh, but that would be tricky. I gues it could be done with PowerShell and VBA, but I leave that exercise to you 😁
I have a hammer and everything looks like a nail. And my hammer is called Excel.
With excel I would folow these steps:
- create a Power Query "API Data" to get the API data
- Load the result into an excel table which creates a table named API_Data
- Create a query "Collected Data" on the excel table API_Data
- And now the tricky bit:
- Load the "Collected Data" query ALSO to a table in Excel to get the excel table Collected_Data
- Change the Colledted_data to use the table Collected_Data
- Yes! It is self referencing
- Remove the excel table API Data.
- Finaly Change "Collected Data" query to combine it with "API Data"
- Change "API Data" to only return data that does not alredy exist in "Collected Data"
- Change "Collected Data" to do an "Append Queries" of "API Data"
You may think this reults in a circular reference, but it doesn't. This is because the Excel Table is NOT the Power Query...
Have fun! I don't know if you want to automate the refresh, but that would be tricky. I gues it could be done with PowerShell and VBA, but I leave that exercise to you 😁
Hi
Thanks everyone for their ideas. PwerQueryKees idea was most suitable for me and did not require any extra learning.
One consideration is that autosave option should be disabled during data refresh because onedrive is constantly locking the file. Otherwise it worked well.