Internal Dataflow Error on Power BI Dataset Refreh

[sjrrkb123]

I am posting here about a recent issue that has been popping causing dataset refresh failures. The error message includes an error called an "Internal Dataflow Error" as the cause of the refresh failure. 

Example:

Data source error:	{"error":{"code":"DM_GWPipeline_Gateway_MashupDataAccessError","pbi.error":{"code":"DM_GWPipeline_Gateway_MashupDataAccessError","parameters":{},"details":[{"code":"DM_ErrorDetailNameCode_UnderlyingErrorCode","detail":{"type":1,"value":"-2147467259"}},{"code":"DM_ErrorDetailNameCode_UnderlyingErrorMessage","detail":{"type":1,"value":"DataflowInternalError"}},{"code":"DM_ErrorDetailNameCode_UnderlyingHResult","detail":{"type":1,"value":"-2147467259"}},{"code":"Microsoft.Data.Mashup.ValueError.Error","detail":{"type":1,"value":"DataflowInternalError"}},{"code":"Microsoft.Data.Mashup.ValueError.Reason","detail":{"type":1,"value":"DataflowInternalError"}},{"code":"Microsoft.Data.Mashup.ValueError.RequestId","detail":{"type":1,"value":"4e91895b-29cf-4fb6-96a2-3d00ff683206"}},{"code":"Microsoft.Data.Mashup.ValueError.RequestUrl","detail":{"type":1,"value":"https://wabi-us-north-central-e-primary-redirect.analysis.windows.net/metadata/v201606/cdsa/dataflow..."}}],"exceptionCulprit":1}}} Table: Dates - Bridge Date Category.
Cluster URI:	WABI-US-NORTH-CENTRAL-E-PRIMARY-redirect.analysis.windows.net
Activity ID:	1a9ad50e-e0ba-4347-87ea-66ba5c97b6d2
Request ID:	1fa05856-84e3-49d2-b27d-9c3255f7cbb3
Time:	2022-05-25 12:05:18Z

This error has occurred on exclusively dataflow based datasets, mixed one with SQL Server (and/or MySQL) and dataflows, but not ones lacking dataflows (which makes sense). The first time this occurred was at the end of March 2022 and has been occurring intermittently and without any apparent pattern. I suspect it is linked to a current know issue in Microsoft regarding dataflow refreshes in Premium Capacity: Dataflow Refresh Issues 

I had hoped someone else in this community may have experienced this issue and could provide steps toward resolution. I currently have a ticket open with Microsoft Support but they do not know the cause.

[v-easonf-msft · Accepted solution]

Hi, @sjrrkb123 

I got similar feedback on this issue.

The cause of this issue is due to a known intermittent issue in Premium Gen2 workloads, including dataflows(ICM:297756751)

The issue is also listed in PowerBI Support forum.

https://powerbi.microsoft.com/en-us/support/

Known issue - Long running, failed or stuck dataflow in Premium Gen2 

---

The current advice is to change the workspace to PRO and back to Premium if the problem keeps occurring.

It is recommended to look into reducing the number of entities within a single dataflow .

Best Regards,
Community Support Team _ Eason

[m-colbert]

This is now happening to us. We were expeirncing random cancellations a few months back. But as of this morning, every single dataflow fails. This issue has been going on since January (per prior support call). Gen 2 has delivered problems, and none of the benefits. We are heavily invested in dataflow usage and I now need to rethink that strategy. The recommneded workarounds are not possibleand way to numerous to keep up with. Just fix it already or put it back that way it was.

[sjrrkb123]

@m-colbert we have been dealing with this since Gen2 upgrade in December 2021. We have had a ticket open with MSFT from March 2022 regarding this. The only current fix is to change the workspace containing the dataflows from Premium capacity to Pro, wait 10minutes, then switch back to Premium. This sometimes fixes it but not always and is the only known way to address the problem.

According to MSFT, the fixes are coming in two waves, June 30 and July https://powerbi.microsoft.com/en-us/support/ 

[m-colbert]

@sjrrkb123

Thanks for the reply. Our ticket is still open as well. I did see that notice this am when I checked for any service outtages. Got MS back on a call just now and magically it all started working again... figures. Was down for about 12 or so hours. Happy it's back though.

Question for you, for the premium workspaces that you've downgraded to Pro and then assigned back to capacity, did this impact any dataflows with linked and/or computed entities? IOW, did downgrading lose some premium only settings?

What about incremental refresh settings? I believe when you downgrade, it drops the partitions and forces a full refresh. Then assigning back to capacity you would need to restablish incremental settings, correct?

For directquery to dataflows, do the reports using the dataflow stop working temporarily and then it's OK after putting back on capacity?

I'm trying to establish what addtional work would come along with the downgrade, wait 10 min and reassign process. 

Thanks again!

[sjrrkb123]

Question for you, for the premium workspaces that you've downgraded to Pro and then assigned back to capacity, did this impact any dataflows with linked and/or computed entities?

- From my experience, this did not affect any linked/computed dataflows we use in this workspace. I cannot speak to what would happen if you have dataflows linked to these dataflows in other workspaces. 

IOW, did downgrading lose some premium only settings?

- You will lose any premium only settings like incremental refresh or direct query dataflows. Again, due to our current data warehouse being full refresh, we are not using incremental dataflows. We do not currently have a use for direct query dataflows either. That said, both of these are listed as premium features so I would expect some interruption. I am not sure to what degree the effect will be.

What about incremental refresh settings? I believe when you downgrade, it drops the partitions and forces a full refresh. Then assigning back to capacity you would need to restablish incremental settings, correct?

- Again, we dont use incremental refresh so I am not much help.

For directquery to dataflows, do the reports using the dataflow stop working temporarily and then it's OK after putting back on capacity?

- To my knowledge, yes they would be interrupted but again we are not using direct query dataflows. Sorry 😞

I would suggest testing out the effect on these premium features in some down time for your organization. 

I wish I could be of more help.

[m-colbert]

@sjrrkb123 

No worries, I appreciate the feedback. I plan to do the testing just wanted to leverage your knowledge if you had already been through it.

Thanks again for your reply. I will post info back after my testing.