Forum Discussion
What about too much data
hi all
When I was using power bi Desktop, there was a data source with about 500,000 lines, which made my data refresh slow. Does anyone know what to do?
I don't need all these data. Is there any way to intercept them?
Best Regards
alan-li
Thanks :)
There are difinity ways to filter it, based on the top N records. filtering based on a field.
you can also apply grouping and aggregations on the dataset to make it smaller if you want.
CheersReza
5 Replies
500K is not a lot of rows.
I had 500M rows with Power Query with no problem.
I believe either the data source is slow that you feel 500K loads slowly. Or you have done some complex set of transformations.
However, if you don't need all the data, there are ways to filter the data and load only parts that you want. For example, if you have a date field, you can only filter data for the previous periods (last 2, 3 years/quarters/months etc)
hi reza_rad
Not many in this day and age. But it's growing by tens of thousands every day, so I was wondering if I could filter some data
Best Regards
alan-li
Thanks :)
There are difinity ways to filter it, based on the top N records. filtering based on a field.
you can also apply grouping and aggregations on the dataset to make it smaller if you want.
CheersReza
hi reza_rad
Big guy, I have been reading your book, thank you very much for sharing
Best Regards
alan-li