Handle Big csv DataFile

[DimaGer]

Hi guys, 

I have a 2GB csv file.

It consists of two columns, products (text) and product group (text). There are about 60 product groups and they can contain up to 20 million unique products.

1) I would like to separate the products per product group into separate tables or simply
2) get a table which shows about 10 random products per product group. I would then export these to a file.

Can someone give me a tip, I fail already with the representation of e.g. the first 10 products per group in a visual, to export them now in a separe Table.

Thanks a lot

[Anonymous]

Hi @DimaGer ,

Whether your problem has been resolved? If yes, could you please mark the helpful post as Answered? It will help the others in the community find the solution easily if they face the same problem with you. Thank you.

Best Regards

[Anonymous]

Hi @DimaGer ,

Do you ultimately want to create reports in Power BI? If so, given the large amount of data you have, importing it directly into Power BI may cause problems. Also, there is a limit on the size of the dataset when uploading to Power BI Service.

Maximum dataset size

You may consider reducing the size of the csv file first as much as possible(for example: delete the unnecessary columns, blank rows etc.). Then import it into Power BI and create a slicer and a table visual to achieve your two needs. I created a sample file, you can see the attachment for more details.

Otherwise, why don't you handle it inside Excel directly?

Best Regards

[amitchandak]

@DimaGer , Try a new table 

filter(
Addcolumns( Summarize(Table, Table[Product], Table[Product Group] , "_val", sum(Table[Value])) ,"_rank", rankx(filter(Table, [Product Group] = max([Product Group])), [_Val],,desc, dense)), [_rank] <=10)

[DimaGer]

@amitchandak 

sorry, dont get it. I am very new to Data and Power BI. i got the following Error Messag:

It says, "cant find Table with the Column [Group]". 

Maybe it helps by providing more details? 

I already loaded the csv File in Power BI Desktop. Now i see the Table which name is "Data" with its two columns Product and Group. 

Thanks