Forum Discussion
API connection, time series analysis issue
Choose a balance between reducing the strain on the source system (pull only the data that you need) and your maintenance effort (DB management) . Instead of a db you can also consider CSV or Parquet files for your storage.
Thanks a lot for patience..
The APIs are connectet to pos/erp software, which has it's own database, so deffinetly they're guaranteed to produce the same data reliably.
So, if there's let's say 50 different api calls each day, that makes 50 different tables in powerbi? If all the data (maybe 5-6 years of data) is pulled from 50 api calls each day, is it ok? Can pbi handle that?
I'm still thinking about database, that it might be more elegant solution, because I could make db schema only with data I need for my reports, and then pull the data from APIs selectivly to the db. Correct me if I'm wrong.
Choose a balance between reducing the strain on the source system (pull only the data that you need) and your maintenance effort (DB management) . Instead of a db you can also consider CSV or Parquet files for your storage.
Hm.. CSV sounds like interesting idea..
If I got this right: Make scheduled API call each day and save it as csv file in some shared folder (google drive for example). Use that folder as data source for the table in PowerBi. Each day when new csv shows up in that folder it's appended to that table in pbi. If there's some change in previous data, I can manually make api call for that day and change that day's csv file. Sounds right?
I guess auto refresh shouldn't be an issue since it's possible to do it if data is in shared folder?Correct. CSVs ingest very fast, so you can do a full load of all CSV files in that folder each time if you want to minimize the maintenance effort.