One row from a dataflow multiplies in the dataset

[AliceW]

Hi guys,

I'm wondering if I'm doing something wrong.

So, I have a dataflow on Opportunities (created using Salesforce Objects). To keep it simple:

Opportunity ID	Last Modified Date	Value
1	2020-05-15	150
2	2020-05-15	200

In the dataflow, I see each Opportunity ID one time.

I set the incremental refresh based on 'Last Modified Date':

- Store rows from the past 5 years;

- Refresh rows from the past 1 day;

- Detect data changes based on 'Last Modified Date'.

I go back in the dataflow, and the Opportunities are still (correctly ) displayed once.

I then connect the dataflow to a dataset. After a couple of days, I notice the dataflow includes SEVERAL lines for the same Opportunity. It kept all 'Last Modified Dates' from the refreshes.

The dataset does NOT have incremental refresh.

My solution was, in the dataset Power Query, to:

1. ORDER BY DESC Last Modified Date and;

2. Remove duplicates.

It seems to be correct, but:

1. Does it really work for all cases?

2. Surely there's a better way to handle things?

Big thank you in advance!

Alice

[v-xicai · Accepted solution]

Hi @AliceW ,

From your description, the Incremental refresh for Dataflow fails to refresh, right? You may refer to the article : Using incremental refresh with Power BI dataflows.

Using incremental refresh for Power BI dataflows requires that the workspace where the dataflow resides be in Premium capacity to run, and the data source ingested into the dataflow must have a datetime field on which incremental refresh can filter.

Best Regards,

Amy 

Community Support Team _ Amy

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.