vNet Gateway - Intermittent refresh issues: DM_GWPipeline_UnknownError

[jpretorius]

When conducting scheduled refresh of datasets using the Power BI vNet Gateway we are seeing intermittent refresh failures with the following error message:

{"error":{"code":"DM_GWPipeline_UnknownError","pbi.error":{"code":"DM_GWPipeline_UnknownError","parameters":{},"details":[]}}} Table: <TableName>

• All network connectivity tests, conducted via the Power BI Service for the vNet gateway, pass
• The datasets can be successfully refreshed when triggered manually from the Power BI Service
• At most, only 3 datasets are refreshing simultaneously against a single datasource
• Datasets are relatively small (>= 50MB  <= 2GB)
• All datasets use Synapse Analytics SQL Serverless datasource
• All datasets would refresh succsessfully (on schedule) prior to moving to the vNet gateway

Please advise.

[Anonymous]

Hi @jpretorius 

Please check your dataset settings , if there is an error like  " Query contains unsupported function. Function name: Resource.Access " , it will be a known issue . Engineers are investigating the issue and an update will be provided soon. 

Best Regards,
Community Support Team _ Ailsa Tao

[jpretorius]

Hi @Anonymous,

I don't see that message in the dataset settings.

Could it be related to Synapse Analytics Serverless Pool inactivity deactivation? 

https://learn.microsoft.com/en-us/azure/synapse-analytics/sql/resources-self-help-sql-on-demand?tabs=x80070002#sql-pool-is-warming-up 

[jpretorius]

I did some additional testing, the issue does not appear to relate to Synapse Analytics Serverless Pool inactivity deactivation i.e. when the Synapse Analytics Serverless Pool is warmed up it's possible to reproduce the error.

I have managed to produce the error by doing a manual refresh, it appears to be directly related to concurrency.

• If I refresh multiple datasets at once the error occurs.
• If I refresh each dataset individually no error occurs.

To clarify, this only happens when using the vNet gateway, I can refresh all datasets simultaneously without the vNet gateway but get the below error when refreshing the datasets simultaneously via the vNet gateway:

"error":{"code":"DM_GWPipeline_UnknownError","pbi.error":{"code":"DM_GWPipeline_UnknownError","parameters":{},"details":[]}}} Table: <TableName>

[Anonymous]

Hi @jpretorius 

I'm so glad that you were able to identify the problem through testing, given your current problem, data updates can only be achieved by manipulating the refresh of a single data set . 

The virtual network (VNet) data gateway is currently a Preview feature, and may not be as perfect as it could be in the parallel refresh section. From the available documentation, it seems that there is no refresh related introduction. 

By the time this feature is officially rolled out, the problems you're experiencing should be fixed. Let's all look forward to it!

Best Regards,
Community Support Team _ Ailsa Tao

[Seanan]

Hi @Anonymous,

Thanks for sheding some light on this issue as I am also experiencing this issue across two of our production workspaces. In some cases it has failed four times in a row causing the scheduled refresh to automatically turn off, which as you can imagine is not ideal.

Do you happen to know roughly when the vNet feature is scheduled to be released?

[jpretorius]

@Anonymous 

I'm finding it incredibly unstable, I'd go as far as saying it's not usable at this point unless you are doing programmatic refreshes (XMLA, REST) of each dataset. 

As you say @Seanan, with scheduled refresh you get regular failures (more failures than successes) and your scheduled refresh settings get disabled regularly. 

[jpretorius]

@Anonymous Can you please confirm these findings have been passed to the product team? Rather than us just assuming they should be fixed once out of preview it would be comforting to know that the issues have been logged and are tracked under an internal ticket.