Forum Discussion

SaiK's avatar
SaiK
Icon for Helper I rankHelper I
1 year ago
Solved

100 GB data on AWS Workspace

I have approx 100GB dataset in Power BI. I am importing this data from MySQL DB.An issue here is am unable to import the entire data because of the size.How do I workabout this issue?
I am also using AWS Workspace so I have memory and space restrictions. I want to publish the 100GB data to power bi service. What should be my approach? How do I go about this?

  • Pre-aggregate means to summarize the data using GROUP BY instead of loading all rows. For example instead of loading by transaction id or transaction lines, group the rows by a certain column.

  • SaiK's avatar
    SaiK
    1 year ago

    Ohh yess, that is something I could do. Instead of applying the measures after importing shall apply GROUP BY before importing. Thanks a lot!!

17 Replies

  • Hi SaiK ,

     

    Not sure of what is the granularity of your data but believe that the best option is not to do import mode but to do Direct Query mode, that way the dataset will be kept to the meta data instead of copying the data to Power BI, this is a litle bit slowrr because it will do a query to your MySQL DB when you run a report but allows to minimize the size of the data in Power BI service.

    • SaiK's avatar
      SaiK
      Icon for Helper I rankHelper I

      I am importing a table from the database. It has 8 columns. I did try the DIrectQuery method with the help of MariaDB connector however it was too slow. Even applying measures was taking more than 5 minutes.

      • MFelix's avatar
        MFelix
        Icon for Super User rankSuper User

        What is the number of rows you are working with?