Forum Discussion

decarsul's avatar
decarsul
Helper V
5 years ago
Solved

PC crashes when doing a left join

Gday all!

 

This week i've been having an issue with a powerquery left join.

I have 2 tables, table 1 contains an id that corresponds with a unique id in table 2.

Now i want to join just 1 column in table 1 that exists in table 2.

 

Both dataflows are roughly 500mb each.

But when i join them, all my 32GB of memory gets eaten in like 1 minute, which results in basically crashing my entire system.

 

Does anybody have any idea how this could be? (i've removed null values from both id columns, so at most, it should be a 1 to many relation).

Need Help
Tips and Tricks
  • aj1973's avatar
    aj1973
    5 years ago

    Ahh okay Merging is not joining. now I understand and for sure its a size issue that you are facing.

     

19 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable
    5 years ago

    Try doing it on Power BI Service, or through Power BI Dataflows in case you have a Premium capacity or Pro License. That may reduce the usage of your memory. 

    Other options would be filtering the tables by some criteria/range and then appending them. Not efficient, but still a workaround. 

    • decarsul's avatar
      decarsul
      Helper V
      5 years ago

      The datasource are PowerBI Dataflows in a premium environment. And i have a pro license.

      Yeah . . . i already skimmed as much as i could, this crashing is also the reason i am only trying left join 1 columnn instead of 8 (which is the bare minimum i need).

       

      Ill try exporting the dataflows to 2 csv files and see i if can join them that way. If that would work, it should point to an issue within the DataFlow itsself, no?

      • aj1973's avatar
        aj1973
        Community Champion
        5 years ago

        Sorry I didn't get your scenario! You have 2 dataflows, you call these dataflows to your Desktop and you pick different entities from these 2 Dataflows. In your model you try to join (relationship) 2 entities from those 2 dataflows and your system crashes?

        Is this your issue?

         

  • aj1973's avatar
    aj1973
    Community Champion
    5 years ago

    Hi decarsul 

    You are trying to connect 2 Dataflows with 500MB each, correct? well that's a 1GB of data for one Dataset which is not possible for a Dataset.

    Your system must have known the truth lol.

     

     

    • decarsul's avatar
      decarsul
      Helper V
      5 years ago

      Hello Amine,

       

      hmmm . . . well that <insert cuss word here> . . . 

      I wish it would give me a warning or something, instead of completely locking up my system tho.

       

      oh btw, does this limit count only for free users, or also for pro users?

      Because i am the latter.

    • decarsul's avatar
      decarsul
      Helper V
      5 years ago

      To followup my question. I'm asking because of this post made in 2018.

      According to this, i can use and upload up to 10GB

       

    • decarsul's avatar
      decarsul
      Helper V
      5 years ago

      And also,

       

      Does this apply only to the online workspace, or also to the desktop client?

      • aj1973's avatar
        aj1973
        Community Champion
        5 years ago

        Answering all your 3 questions:

        1. Limits count for free and Pro licenses
        2. The 10GB size is the limite for a workspace not for a Dataset. one workspace can contain many Datasets, one dataset can't be more than 1GB size, size of all Datasets can't go over 10GB in A workspace
        3. Same answer as question 2