Forum Discussion
Loading huge file and RAM overload
Good morning,
I'm through a bit of trouble while trying to load a huge file into Power BI.
This file is an array of JSON objects weigthing 80+ GB in total, so I load it as a JSON object, but after some minutes while the content is being analysed, Microsoft Mashup Service is eating all of my RAM. I have followed advise for optimizing data load in other posts (disabled paralel loads, etc.) but it's still too much for Power BI.
Is there any way that Power BI uses pagination instead of eating all my RAM? Am I doing something wrong, or the simple idea of loading such huge file in Power BI is infeasible?
Thank you very much.
Based on my test, there is not such a option to set a fixed amount of memory in Power BI Desktop. In your scenario, workaound for you is import Json file to SQL Server database. And then connect to this database by using DirectlyQuery mode.
Here are some useful links for you reference.
https://blogs.msdn.microsoft.com/sqlserverstorageengine/2015/10/07/bulk-importing-json-files-into-sql-server/
https://community.powerbi.com/t5/Service/Direct-query-or-import/td-p/98683Regards,
Charlie Liao
1 Reply
Based on my test, there is not such a option to set a fixed amount of memory in Power BI Desktop. In your scenario, workaound for you is import Json file to SQL Server database. And then connect to this database by using DirectlyQuery mode.
Here are some useful links for you reference.
https://blogs.msdn.microsoft.com/sqlserverstorageengine/2015/10/07/bulk-importing-json-files-into-sql-server/
https://community.powerbi.com/t5/Service/Direct-query-or-import/td-p/98683Regards,
Charlie Liao