Forum Discussion

dura45's avatar
dura45
New Member
2 years ago

API call and historical data

Hello everyone,

I'm new to the forum and Power BI, and I have a question for those more experienced than me.

 

I have a series of API calls to make to gather data needed to populate a dashboard. However, due to limitations imposed by the API call, I must make a filtered call based on date. So, the GetSessionByData call is structured like this [URL]/[Date Parameter].

By doing this, I get partial data, but I want to go back at least 1 year. Therefore, I've created a function that sets the [Date] parameter as a variable that takes data from a second table containing only dates from the past year, giving me what I need.

 

However, this violates the API limits set by the provider because every time I refresh the data, it downloads the entire history of the past year. So, I've been thinking about implementing incremental updates. When the user clicks "refresh," it should only download what's missing. But I'm not clear on how Power BI handles the already downloaded data. Does it store it somewhere? If there's a lot of data, am I at risk of exceeding Power BI's storage limit?

 

I'm a bit confused, and if someone could help me understand this better, I would be very grateful.

Need Help
Tips and Tricks

1 Reply

  • AmiraBedh's avatar
    AmiraBedh
    Super User
    2 years ago

    Does Power BI store already downloaded data somewhere?

    Yes, Power BI stores data in a data model. This data model exists within your Power BI file (.pbix) when you are working in Power BI Desktop, and on the Power BI service when you publish your reports. The data model is loaded into memory when you open or interact with your reports.

    Storage Limits : It depends on your computer's hardware. However, if you're publishing your dashboard to the Power BI Service, you need to be aware of service limitations, especially if you're on a shared or free service. Each workspace in the Power BI service has a limit of 1 GB per single dataset and varying limits for data refresh based on your licensing. You might also have a limit on how much data you can refresh in a single day.

    Implementing Incremental Updates:

    Power BI allows for incremental data loads where only data that has changed is refreshed. This usually requires having a DateTime column in your data that Power BI can use to identify new or changed records. If your API allows you to query by a datetime field ( in easier words : "give me all records updated after [Date]"), this would be an ideal solution.