Problems with dataflow refresh - Snowflake source

[JGauci]

I am establishing a new dataflow from a Snowflake source with the refresh/initial load appearing to hang. The issue is occuring on fact tables; dimensions are importing/refreshing fine. A subset of the fact table works with approx 100K rows. My current long running refresh is approx 100M rows which has been running for over 2 hours. 

The Snowflake query history shows the query completed in 25s yet the most recent attempt is still continuing.

The Snowflake connection is going via an on-prem gateway; the dataflow is being established in a Premium capacity workspace.

Are there any tips to see/identify what is causing the refresh to hang?

[Morcreative]

Hi, I'm also facing the same issue with dataflow in powerbi service...fails after 3 or 4 minutes.

Did you find a solution?

Thanks

Greg

[Anonymous]

Hi @JGauci ,

How about the refresh on the Power BI Desktop? 

You also must install the Snowflake ODBC driver on computers that use the Snowflake connector, using the architecture that matches the installation of Power BI Desktop, either 32-bit or 64-bit. Just follow the following link and download the appropriate Snowflake ODBC driver.

You can refer to the document :https://docs.microsoft.com/en-us/power-bi/desktop-connect-snowflake.

You may go through the troubleshooting links to resolve the issue : https://docs.microsoft.com/en-us/power-bi/refresh-troubleshooting-refresh-scenarios

There is also a similar post, please refer to it to see if it helps you.

Solved: Snowflake data refresh extremely long - Microsoft Power BI Community

If you still have this issue for Power BI, you'd better create a support ticket in Power BI Support to get further help.

Best Regards

Community Support Team _ Polly

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[JGauci]

Hi _Polly, Thank you for replying. This issue is in relation to data flows in the Power BI service hence bypassing desktop. I have no issue importing and using data within the desktop however that is not the use case I need to answer for. Our intention is create the data flows from Snowflake for the enterpise data assets and have the report builders connect to the data flows for their reports/visualisations. The initial import did finally work after 12 hours:

Incremental refreshes have since been working well - the main issue will be for initial/full extract. The above was using our pre-prod environment and with the fact table restricted to 1 year/~100M rows. 

I will need to review my approach to load the full history data in perhaps breaking this up over several data flows/removing earlier years unless there is another approach to getting the full history in. I would rather avoid timeouts/failures at 50% in etc.

[Anonymous]

Hi, I'm also facing the same issue with dataflow in powerbi service... its sales order line data for couple of years, millions of record, usually take ~1hour to refresh but often it fails...

Did you find a solution?

Thanks

Gur