Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
7 years ago

Using Union in DAX vs Append in Power Query: Impact on Storage in Gateway

Hi Experts,

 

I have an issue of a table exceeding the 10 GB limit on Gateway. I know the heavy table that I have has 49M records and growing. I need all the records. No. of fields are only 10 single key and Normalised table.

 

Anyway the question is if I split this table into multiple queries and then do a UNION in DAX to create final table; will it actually reduce the size of the table at the gateway level?

 

Another option is to combine it using Append in Power Query, but my gut feel is that the resulting table is going to count towards storage at gateway level.

 

Any suggestions are highly appreciated!

 

Regards

Dev

Need Help

1 Reply

  • Anonymous's avatar
    Anonymous
    Not applicable
    7 years ago

    Anonymous,

    What is you data source? The 10GB limitation is placed for each table. After you split the table into multiple queries and then do a UNION or append to create final table, you may still get refresh error as the final table can exceed the 10GB limit.

    To solve the original issue, remove unused columns,reduce the use of highly constant, long string values and instead using a normalized key. And change data type in your data source(INT-> tinyint or Smallint, Decimal -> smallmoney or money), for more details, please check iJuancho's reply in this similar thread.

    Regards,
    Lydia