Power BI Dataflow - Power BI Dataset

[webchris]

Hi there

I am currently facing issues with Dataflows in combination with Datasets.

In my Dataflow I had, a while ago, an property with 3 different records. I changed the queries, rebuild the logic and updated the Dataflow, so that now this property has only 1 record (which is fine). When I connect a PBIX Dataset to this flow and query the data, I keep getting the property with the 3 different records.

Before you ask:
- I refreshed the Dataflow, I refreshed the Dataset, still 3 records

- Then, I added a custom column, refreshed the Dataset: The custom column appears, but the 3 records are still there

- I created a new Dataset, connected to the Dataflow, still 3 records

- I created the Dataflow from scratch, then it worked - only 1 record
  (but now comes the funny part: after my 2 weeks holiday -
   the new flow now shows 3 records again in the Dataset (as well as in new PBIX files connecting to the Dataflow)

What is going on there?

We are seeing a similar issue in another Dataflow as well.

Thanks for your help

Christian

[Joshrodgers123]

I've been seeing many weird issues with dataflows recently using the staging dataflow-> transform dataflow-> datasets approach. Tables are not getting refreshed when changes are made, or are left completely empty. This seems to be occuring mostly when used with deployment pipelines. I've had to reopen the dataflows and save them again to get any changes to actually be applied during refresh.

[webchris]

Thanks for your idea, but that did not work... To check whether I am seeing ghosts, I opened and saved the flows several times, also with my colleagues to double-check.

And, although the flows are in a deployment pipeline, they are in the DEV stage, so they have not been deployed there.

Thanks, I'll keep waiting for ideas...

Christian

[webchris]

I am querying results for Properties (in this case Buildings). In the Dataflow, I get 1 record as a result for a specific Property (Building). This Property is allocated to one Fund (therefore 1 record).

A while a go that Property was allocated to 3 Funds, which was okay at that time, but is no longer. 

But when I query this Dataflow, I get the results which was valid a while ago. Although I refreshed everything several times...

[lbendlin]

What is the data source for your dataflow?

[webchris]

First Dataflow has Salesforce as a Source.

This is an "old" fashioned Dataflow which works properly (no problem as in the other topic) - this is my daily "staging".

The second flow queries the first Dataflow and contains some transformations and calculations. So far, that ran fine and did not cause issues (untill 3.5 weeks back).

The Dataset queries the second flow.

[lbendlin]

Too many moving parts, in my personal opinion. I don't think dataflows are providing any meaningful benefit for a Salesforce data source.  For example, I have a 30 GB dataset with incremental refresh fed from a SFDC object query.  I cannot imagine what the effort would be to have two dataflows feeding that dataset.

[webchris]

Why is that "too many moving parts"? This is one way to do ETL, isn't it? 

[webchris]

As I mentioned before: This is not a Salesforce issue. 

The same issue happened in another Dataflow-Dataset Combination where no Salesforce is included.

[lbendlin]

Please explain what you mean by "property"  and "record" .  Did you mean queries and rows?