Unable to get updated data from dataflow

[Justinphilliphk]

Hello support, I am having a question that after the refresh of my dataflow, sometimes the reports are unable to receieve the lastest data. So I do some checking by myself. I have checked that in dataflow quickview that the latest data did exist, while when I am trying to import the data into new report, those latest data is not able to import into the report. May I know the reason of why this is happening?

[v-xiaoyan-msft]

Hi @Justinphilliphk ,

We have seen on the internal platform that other engineers have reported this issue and submitted an icm.
The engineers on the product team are now aware and will do their best to resolve the issue. I will update here if there is any progress, so please be patient.

Best regards.
Community Support Team_Caitlyn

[analyticsyc]

I seem to be having a similar issue. My dataset can load in the most recent data from my dataflow, but when I create a new column in my dataflow it is not showing up in my dataset on PowerBI desktop. I am using an ETL process with dataflows as documented here (https://learn.microsoft.com/en-us/power-bi/guidance/powerbi-implementation-planning-usage-scenario-a...). Since I perform all data transformations in an intermediary dataflow, the dataset on desktop should just load in the new column without any issues. I've also tried creating a brand new PowerBI desktop dataset, and even with loading in the dataflow from scratch, the new columns do not show up. I have refreshed the dataflow multiple times, and I can see the added column in the online preview function. 

@v-xiaoyan-msft or anyone else, is the issue I'm describing known? Is there any workaround? Or any ETA on a fix?

EDIT

I have figured out a workaround for my own issue. Despite refreshing the dataflow in PowerBI Online, the only way to get the new columns to push to my PowerBI desktop dataset was to refresh the preview in PowerQuery Online (where dataflows are created). Then I refreshed the dataset itself from the workspace. Then refreshed the dataset on PowerBI desktop, and the new column was pulled into the dataset. Not too difficult of a workaround, but I can't say it makes much sense as to why I have to do it. 

Thanks for the guidance. 

[analyticsyc]

I am experiencing the same issue as @Justinphilliphk now. When I refresh my dataflow on PowerBI online, new data is stored in the dataflow and I can see it in the online preview. However, that new data is not brought into my dataset on PowerBI desktop when I refresh. 

I've found if I go into the dataflow query editor and refresh the preview, save changes, refresh the whole dataflow on PowerBI online, refresh the datset on PowerBI desktop, then the new data is imported into my model. This process is very manual though and is not sustainable for the multiple datsets I have and the constant refreshes they require. @v-xiaoyan-msft Please keep us updated - this issue is making it almost impossible to use dataflows for our company. 

[Justinphilliphk]

Hi @v-xiaoyan-msft 

It that any new updates for the case? Just wanna know is there are any process for the problem or how long more is need for your product team to fix the issue.

[rdnguyen]

I had 2 cases like that. The newest case showed that data could be disconnected between Dataflow upstream and Dataflow downstream within the same workspace even.

When I do a small modification in the Dataflow upstream, such as sort, then save change, then undo change and save to revert any modification, then refresh; that time data will flow through. However, this is still a manual workaround which I am not going to call it solution.

Worse, users in my company started distrusting reporting system by Power BI and tend to blame their fault on data first thing citing data is not updated. What a headache!

[analyticsyc]

@v-xiaoyan-msft do you have any update? Anything would be greatly appreciated seeing as how this feature fully doesn't work right now

[analyticsyc]

@Justinphilliphk @rdnguyen Try updating PowerBI desktop to the newest April release. Then republishing the dataset to PowerBI online. This seems to have fixed my issue and I'm cautiously optimistic it is a permanent fix - going through testing now and the next few days

[rdnguyen]

@analyticsyc keep me posted. Very interesting to learn from you.