What about too much data

[Anonymous]

hi all

       When I was using power bi Desktop, there was a data source with about 500,000 lines, which made my data refresh slow. Does anyone know what to do?

       I don't need all these data. Is there any way to intercept them?

Best Regards

alan-li

[reza_rad]

500K is not a lot of rows.

I had 500M rows with Power Query with no problem.

I believe either the data source is slow that you feel 500K loads slowly. Or you have done some complex set of transformations.

However, if you don't need all the data, there are ways to filter the data and load only parts that you want. For example, if you have a date field, you can only filter data for the previous periods (last 2, 3 years/quarters/months etc)

[Anonymous]

hi @reza_rad 

Big guy, I have been reading your book, thank you very much for sharing

Best Regards

alan-li

[Anonymous]

hi @reza_rad 

Not many in this day and age. But it's growing by tens of thousands every day, so I was wondering if I could filter some data

Best Regards

alan-li

[reza_rad · Accepted solution]

Thanks 🙂

There are difinity ways to filter it, based on the top N records. filtering based on a field.

you can also apply grouping and aggregations on the dataset to make it smaller if you want.

Cheers

Reza

[Anonymous]

hi @reza_rad 

thinks a lot , 

Best Regards

alan-li