Refreshing Dataset which is connected to the Dataflow with Incremental Refresh

[denis163]

Hi All!

We use Dataflows with incremental refresh as a primary source for our self-service users. Refreshes of Dataflows work really fast and we don't have any issues with that, but refreshes of Datasets which are connected to that Dataflows are much slower (we didn't set up incremental refreshes for datasets, because we received some errors and thought that incremental refresh for dataset and dataflow at the same time isn't supported).

I didn't find any information about how exactly happen refreshes of these kind of Datasets and I hope that you can help me with that.

For instance, we have:

1) Dataflow with incremental refresh - rows are stored from the past 2 years, refresh only last 7 days data.

2) Dataset without incremental refresh which is connected to the Dataflow with settings above.

While scheduling refreshes of our Dataset in Power BI Service - will it only update new data in the last 7 days from Dataflow or will it refresh the whole data which is stored in Dataflow?

Thank you in advance!

[MAwwad · Accepted solution]

When you refresh a dataset that is connected to a dataflow with incremental refresh, Power BI will perform a full refresh of the dataset. This means that Power BI will query the entire data stored in the dataflow, not just the new data that was added since the last refresh. The data stored in the dataflow is considered the source data for the dataset, and a full refresh is necessary to ensure that the dataset is up to date with the latest changes made to the dataflow.

It's worth noting that the time it takes to refresh the dataset will depend on the size of the data stored in the dataflow and the performance of the Power BI service. If you find that the refresh is taking too long, you may want to consider using incremental refresh for the dataset. Keep in mind that using incremental refresh for both the dataset and the dataflow at the same time can sometimes result in errors, so it's important to test and validate that this configuration works properly before enabling it in your environment.