Data Refresh Storage and Traffic Concerns

[Greg_Davurse]

I have a couple questions concerning the data refresh when connecting to an on-premise SQL server/DB. 

Once the report has been made and the Refresh Schedule has been set, how is the data refreshed and stored? How is the Power BI Service updating the data and where is it stored? For Example: if I have 10,000,000 records, does the PBI Service refresh all 10,000,000 records and then some if additional records have been added? If so, this brings me to my next main question...Will this/can this cause latency issues on the server do to a large amount of data traffic since Power BI is pulling a large amount of data from the Server/DB, and if so, is there any documention on best practices or does anyone have any advice when setting up these refreshes?

I apologize in advance if I've made any assumptions in my question. Please correct if I have misinterpretted anything. 

[Greg_Deckler · Accepted solution]

If this is an on-premise database, then Power BI Service uses an on-premises gateway to perform the data load. Currently, Power BI does not have incremental load so all 10,000,000 records plus additional would be refreshed. If that original data never changes and latency is a concern what I would do would be to create a view in your database and 2 or 3 queries. Essentially, create one query that grabs all of your original data. Then, create a view that only shows new data. Create another query to grab that data from that view. Append the two queries. This way, you will only refresh a subset of your data and not all 10,000,000 records every time. 

You could also use Direct Query, which pulls the data real time instead of importing.

[Greg_Davurse]

Thanks! Your explanation helps and answers my question.