Importing 6 billion Data tables from Snowflake

Hi Anonymous ,

Despite of the executionspeed of Snowflake itself, the data still has to be transferred to your local machine. A table with 6 billion rows, that is a massive amount of data to transfer from Snowflake onto your laptop or Power BI service. To give an example: if every row holds only 1Kb of data (which is not a lot to be honest), then your data size would mount to 6,000,000,000 / 1,024 (=mb) / 1,024 (=gb) = 5,722 GB. There would be some smart compressing be going on but just to illustrate that the amount of time of getting results from a query of an external source is not just the query execution time; it is also the transfer time. Especially in cases like this. 

If this is too long for you, you might want to look into incremental updates (but that requires Power BI premium);

https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh 

Hope this gives you some clarification as to why this takes longer than you expected 🙂

Kind regards

Djerro123

-------------------------------

If this answered your question, please mark it as the Solution. This also helps others to find what they are looking for.

Keep those thumbs up coming! 🙂