Forum Discussion
Snowflake data refresh extremely long
I have built out fairly involved dashboard which is pulling in YTD data from our Snowflake platform. 12 seperate queries some pulling in as many as 18M rows of data. As you can imagine the refresh process takes some time, about 3 hours from the desktop. So I have set up incremental refresh and a private gateway to push this to the service portal. Now it begins the refresh, I can see the jobs running in Snowflake but they time out. Is this due to the inital load being so heavy? Will it get lighter once the initial YTD data is in and I'm just running 10 days incrementally? For my initial load, is there a way for me to break this up into maybe 3 month batches? Need help, our Snowflake platform and PBI don't seem to be playing well together.
Hi Anonymous ,
“As you can imagine the refresh process takes some time, about 3 hours from the desktop. ”
I think you need to reset your incremental refresh strategy and optimize your data model.
Best regards,
Lionel ChenIf this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
2 Replies
Anonymous the initial full load will take longer, I'm not sure what you mean by 12 different queries, are these different tables or the same on? 18 million is not that many rows. Are you using direct query?
Hi Anonymous ,
“As you can imagine the refresh process takes some time, about 3 hours from the desktop. ”
I think you need to reset your incremental refresh strategy and optimize your data model.
Best regards,
Lionel ChenIf this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
