Need help in finding way to accumulate data in Power BI

[Anonymous]

I need help in finding way to accumulate data with source as web. I am currently using power query in excel. There's a query fetching data from web (Manufacturing data) for last 5 days. I load the data in excel then for next refresh, i have helper query which fetches data from the same table where web query loaded data. these 2 query merge and load it to same table.

Query A : fetches data from web for last 5 days & appends data from query B. Query A loads to Table1.

Query B : fetches data from Table 1.

So, in this case, i get data accumulated in Table 1.

How to imitate this use case & get accumulated data in Power BI ? is there any way?

[Anonymous]

See, website has from & to dates in the link, I have used (current date -5) formula in "from days" & (current date) in "to days". All I want is, data should get accumulated with each refresh. lets say, I have refreshed 2 days ago (today is 20th Sept), so I have data from 13th Sept till 18th sept. Now with regular query if i refresh now, all the data will be replaced by new data which is from 15th to 20th sep. But I want earlier data to be there as well.

is it possible via Incremental refresh? Incremental refresh works for Import mode (as it is web data source)? I dont have pro or premium account.

[Joe_Barry]

Then the Incremental refresh approach is the way to do it. Maybe add a column to the table and enter DateTime.LocalNow() this will create a timestamp in Date/Time format of the import. You can then use this column for teh incremental refresh.

Plese read the documention, as will explain exactly what you need to do.

Joe

[Anonymous]

I am using below link in Web.Contents() with RangeStart & RangeEnd for driving dates.

https://webpoint.fiat.com/webpoint/DettaglioCISweek.aspx?tg=TRANSITI&stab=59&stazda=24&staza=24&from...

But in Power BI service, schedule refresh is not working as its a dynamic source - "This dataset includes a dynamic data source. Since dynamic data sources aren't refreshed in the Power BI service, this dataset won't be refreshed. Learn more:". Please help in this case to implement incremental refresh.

[Joe_Barry]

Hi @Anonymous 

To understand correctly, you are getting the data from the Web and saving this initial query result and then requesting more data and appending it to the initial query result?

Copy your Power Query Code to Power BI. Just Right click on the Query name in Excel and Paste into the field on the left for queries in Power BI.

You will want to implement incremental refresh in Power BI.  Follow the instructions here https://learn.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-overview#create-parameters

Please note you will need to have a DateTime column in the table for this to work.

If you have Premium, you could create a Dataflow with the same query and set up the incremental refresh this way https://learn.microsoft.com/en-us/power-query/dataflows/incremental-refresh

Hope this was helpful

Joe