Forum Discussion
Append Data to existing table
Hi Pat,
The incremental refresh has one precondition that is the data should be stored in the source. But it seems the source only has the latest data in your description.
Did you do to improve the performance of refreshing?
Best Regards,
I'm not following you!
I need to connect to Web Data source ( like https://finance.yahoo.com/quote/MSFT) and grab a table and store in a table, the next time I refresh I need to grab new data and append to previously collected records with a timestamp, soemthing like building a qutoa collections over the time.
-Pat
Hi iLikeAzureSQL ,
I would suggest you try the Folder connector. You can store the new table in a folder. You will get all the data combined when you refresh the report.
Best Regards,
There's no way to do that, I've been struggling with the same... it is so easy to do that in QlikView for example with the incremental load. But power BI works differently, it loads all the data again from the source every time it runs, so the only way to achieve this is that you manually create daily external files to then load them in Power BI. There's a way to create a CSV out of Power BI using an R script, but that will only work if you refresh your dashboard on desktop or personal gateway. R scripts are not supported in the enterprise gateway.
Here's how to use the R script in case it may help you, this will create a new file out of the query with the format History_yymmdd.csv
write.table(dataset, file=paste("C:/Inputs/OTC_Vital_signs/ALL_ERP_HISTORIC/History_",format(Sys.Date(),"%y%m%d"),".csv"), sep=",", row.names=FALSE, append = FALSE, col.names=TRUE)
You can add the R script in the query editor, on the Transform tab; add the step at the very end of your query to generate the CSV when it is done with the transformations
