Forum Discussion

majdkaid22's avatar
majdkaid22
Helper V
9 years ago

compress dataset

Hello Community,   I have a Dataset used by a user who every time create new measure or update something in the power query, would refresh the dataset on the desktop and then publish it.    The D...
Need Help
majdkaid22's avatar
majdkaid22
Helper V
9 years ago

Phil_Seamark thanks mate. all the above have been taken into consideration already. we are in the FX Trading and I have tables with over 1m rows a day. 

 

My data sources are SQL, MYSQL mainly 

Phil_Seamark's avatar
Phil_Seamark
Microsoft Employee
9 years ago

If your datasources is MS-SQL, have you considered building a summary table in SQL prior to Power BI rather than reading from raw tables?

 

What is the lowest grain you need to report on in Power BI?  That would be the target level of summarisaion in SQL.

 

 

  • majdkaid22's avatar
    majdkaid22
    Helper V
    9 years ago

    Phil_Seamark that is a long term option which am working on. at the moment I have MYSQL DB which are a bit messy and I don't have the enough resources to structure views in there. 

     

    Guess I will have to try and go on another round of removing unrequired columns...

     

     

    Cheers mate

    • Phil_Seamark's avatar
      Phil_Seamark
      Microsoft Employee
      9 years ago

      Columns that have unique values are good ones to target.  If you can get rid of these then your ability to compress will be improved.

       

      Also if you can pivot data to be vertical rather than horizonal, this will help