Incremental refresh set up

[Anonymous]

Hello Microsoft,

My source is Blob Storage , where I would receive 'n' number of files. From Power BI I try to connect to Blob Storage link and pull those files /tables to power BI desktop. Everyday, I have huge data(10 Crores rows) from each file. I am finding difficult to load data everyday manually and many times my desktop file breaks to load due to huge volume.

Now,  I would like to set up an incremental refresh by creating parameters, how that I could do with Blob storage as a source?
Please, help

[v-jianboli-msft · Accepted solution]

Hi @Anonymous ,

If you want to connect to Blob Storage as your data source:

From Get Data, select the Azure category, select Azure Blob Storage, and then select Connect. 

In Azure Blob Storage, enter the account name or URL of your Azure Blob Storage account. Then select OK.

If this is the first time you're connecting to this account, select either the Anonymous or Account key authentication method to sign into the Azure Blob Storage account.

Then select connect.

If you want to set up incremental refresh, here are some limitations that you need to consider about:

Incremental refresh and real-time data works best for structured, relational data sources, like SQL Database and Azure Synapse, but can also work for other data sources. In any case, your data source must support the following:

• Date column - The table must contain a date column of date/time or integer data type.
• Query folding - Incremental refresh is designed for data sources that support query folding, which is Power Query's ability to generate a single query expression to retrieve and transform source data, especially if getting the latest data in real time with DirectQuery. 

For more details, please refer to:

Azure Blob Storage - Power Query | Microsoft Learn

Incremental refresh for datasets and real-time data in Power BI - Power BI | Microsoft Learn

Best Regards,

Jianbo Li

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.