Forum Discussion

VaibhavK's avatar
VaibhavK
New Member
6 years ago

Need Help- Power Query: Merge 2 Sharepoint online lists

Dear Community,

I have 2 sharepoint online lists 25k and 6000k items that need to combine data and then display in report form. The Primary key is DO_ID. I have created a merge query. But some items are missing. This is some wierd thing that I have observed. What may cause this thing? I have created 2 separate queries, merged them and then uploaded to sharepoint document library. I use excel web part to show data but the reporting is not correct. 

Can anyone list correct steps to achieve this goal?

6 Replies

  • v-yingjl's avatar
    v-yingjl
    Community Support

    Hi VaibhavK ,

    Here are the common steps to connect to SharePoint files and combine them in power bi desktop:

    1. Use the web connector to connect specific files on SharePoint:(URL is like ''https://xxx.sharepoint.com/sites/xxx/Shared Documents/xxx.xlsx" )

     

    2. In power query, use 'Merge query as new'

     

    3. Use your primary key to merge these two tables, default join kind is 'Left Outer'

     

    4. Expand table and remove uncessary columns, close and apply it.

    5. Based on this merged table, you can create reports in power bi desktop and upload to SharePoint.

     

    Please note that:

    Merging requires a common attribute to join on, this ensures that the new attributes are correctly matched within the output. There can be missing values within either dataset provided the attribute being merged upon does not contain missing values.

     

    In conclusion, merge will add columns and only show repeated primary key columns and their values, so it is normal that you will miss other items after merging tables. If you want to add rows, you can try to use append in power query.

    This article and case explain the differences between append and merge in details that you can refer, hopes to help you:

    1. Append vs. Merge in Power BI and Power Query 
    2. merging and appending 

     

    Best Regards,
    Yingjie Li

    If this post helps then please consider Accept it as the solution to help the other members find it more quickly.

     

  • Hello Community, I get some weird values instead of actual values. I get some numbers instead of actual values. What could it be?
    • v-yingjl's avatar
      v-yingjl
      Community Support

      Hi VaibhavK ,

      Could you please consider sharing how did you do it and some screenshots for further discussion?

       

      Best Regards,

      Yingjie Li

  • Hi v-yingjl 

    Query 1

     

     

     

    Query 2 has numbers in plant ID e.g. It needs to have values as below but it gets numbers.

     

       

     

     

     

     

     

    The result has numbers that come from Query 2 having numbers instead of of values.

     

     

     

     

     

  • Hello community, the problematic column is lookup column in sharepoint. How to handle this column in powerbi to get actual value instead of values?