Data refresh blob

[avinrosh]

Hello;

I have a repository of in-house built multiple tools and using application insight we track their usage. The raw data coming from Application Insight is being stored in Azure blob storage and we have put up a mechanism which keeps on continuosly exporting the data to blob.

I have connected my Power BI data source to blob storage. When I refresh my data it take me close to 2-3 hours to get is refreshed since the data to be pulled in from Blob is of the size 400 mb and it will keep on increasing at the data from application insights keeps on aaculmulating.

I want to refresh data everyday and have to spend that much time. Is it possible for Power BI to remember the historical data and refresh the only data which have been accumulated for a day.

Regards

Avinash

[v-diye-msft]

Hi @avinrosh 

Incremental refresh could fix it, provided that you have the premium license.  

Incremental refresh enables very large datasets in the Power BI Premium service with the following benefits:

• Refreshes are faster - Only data that has changed needs to be refreshed. For example, refresh only the last five days of a ten-year dataset.
• Refreshes are more reliable - It's no longer necessary to maintain long-running connections to volatile source systems.
• Resource consumption is reduced - Less data to refresh reduces overall consumption of memory and other resources.

More details please refer to: https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh and the license purchasing info: https://docs.microsoft.com/en-us/power-bi/service-admin-premium-purchase

Best regards,

Dina Ye

[v-diye-msft · Accepted solution]

Hi @avinrosh ,

If my above post helps, could you please consider Accept it as the solution to help the other members find it more quickly. thanks!

Best regards,

Dina Ye