Forum Discussion

alakiran's avatar
alakiran
Regular Visitor
8 years ago

Importing 320 million

Hi,

 

I am trying to import a large data set of 320 million rows into powerbi desktop and it is failing with below error. Any suggestions/ help?

 

Apply query changes

Failed to save modifications to the server. Error returned. There's not enough memory to complete this operation. please try again later when there may be more memory available.

8 Replies

  • alakiran's avatar
    alakiran
    Regular Visitor
    8 years ago

    Hi,

     

    I am trying to import 320 million records to PowerBI desktop and after running for 5 hours, it is failing with the error. Any suggestions or help please?

     

    Apply Query changes

     

    Failed to save modifications to the server. Error returned: There's not enough memory to complete this operation. Please try again later when there may be more memory available

     

    • Phil_Seamark's avatar
      Phil_Seamark
      Icon for Microsoft Employee rankMicrosoft Employee
      8 years ago

      Hi alakiran

       

      Remove any columns you don't need.  Do you have any columns that have highly unique columns?

       

      Do you have any DateTime data values?

       

      What is your data source?

      • alakiran's avatar
        alakiran
        Regular Visitor
        8 years ago

        Yes, I think i selected 6-8 columns out of 30 columns and it is still failing with the error. Yes, There is a datetime column out of those selected columns.

  • Greg_Deckler's avatar
    Greg_Deckler
    Icon for Community Champion rankCommunity Champion
    8 years ago

    What is the data source? Sounds like you need more memory or should switch to Direct Query.

    • alakiran's avatar
      alakiran
      Regular Visitor
      8 years ago

      Azure SQL Server is the data source, Direct Query has a limitation to retrieve only 1 million records.

      • eniX's avatar
        eniX
        Icon for Helper III rankHelper III
        8 years ago

        you need more RAM apparently. Be prepared to the fact that your .pbix-file may exceed the 1 GB size limitation (PBI Premium needed?)

  • zoloturu's avatar
    zoloturu
    Icon for Memorable Member rankMemorable Member
    8 years ago

    alakiran,

     

    320 mln rows is quite big amount. I have some suggestions but firstly please clarify below:

     

    1) How often do you need to run import from Azure SQL? What is the frequency?

    2) I guess you not plan to use this file in Power BI Service. Correct?

    3) How much is your RAM?