Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
6 years ago
Solved

Load distinct excel Sheets from Share point folder

Dear Experts,

 

I would like to load distinct excel spreadsheets from the sharepoint folder. 

In detail: My share point has a folder called 'Customer List'--> inside 'Customer List' folder  -->there are Monthly folders example 'Jan2019', 'Feb2019', 'Mar2019' folders available each folder has approximatly 10 to 15 excel spreadsheets.

 

when i load the data into power BI, it consumes all the spreasheets from Customer List, but i got many duplicates which i want to avoid duplicates please suggest me the solution

the data is like below

Folder-Oct2019

1. 231Tom.xlsx

2. 232Racheal.xlsx

3. 768Kiran.xlsx

 

Folder-Nov2019

1. 988John.xlsx

2. 232Racheal.xlsx

 

I am expecting the end result like below:

1. 231Tom.xlsx

2. 232Racheal.xlsx

3. 768Kiran.xlsx

4. 988John.xlsx

 

please suggest me the solution in powerBI

 

 

Need Help
  • JarroVGIT's avatar
    JarroVGIT
    6 years ago

    Hmm that would require some date magic I guess, based on folder name and file name. For a full logic solution we would need to know what the exact structure is. 

    First step is to create columns based on the filename and/or filepaths, containing just 'data1' rather then 'data1Nov.csv'. Then you can remove duplicates but without the full list of folders/files (or a full list of possible combinations), it would be guessing 🙂

    Kind regards

    Djerro123

    -------------------------------

    If this answered your question, please mark it as the Solution. This also helps others to find what they are looking for.

    Kudo's are welcome 🙂

3 Replies

  • JarroVGIT's avatar
    JarroVGIT
    Resident Rockstar
    6 years ago

    I've recreated an example with a local folder, it would work the same. If you load the folder and have a list of all files, it should look like this:

    Note that data1 and data3 are duplicates, they are in different subfolders in my case. 

    Right click on the column "Name" and select Remove duplicates

    This deletes all subsequent occurences. If you want to keep the last occurence rather then the first, perform some filter and sorting magic before removing duplicates 🙂

     

    From here, you can merge and load all binaries like normal.

     

    Kind regards

    Djerro123

    -------------------------------

    If this answered your question, please mark it as the Solution. This also helps others to find what they are looking for.

    Kudo's are welcome 🙂

    • Anonymous's avatar
      Anonymous
      Not applicable
      6 years ago

      Hi Djerro, 

       

      Many thanks for the kind reply.  nearly reached.. before mark it as solution, I have couple of files with the below format which cannot remove

      example:

      Data1Nov.csv

      Data1Dec.csv

       

      I want to pickup recent distict file which is just Data1Dec.csv.  

      please shed some light.

       

      thanks,

      • JarroVGIT's avatar
        JarroVGIT
        Resident Rockstar
        6 years ago

        Hmm that would require some date magic I guess, based on folder name and file name. For a full logic solution we would need to know what the exact structure is. 

        First step is to create columns based on the filename and/or filepaths, containing just 'data1' rather then 'data1Nov.csv'. Then you can remove duplicates but without the full list of folders/files (or a full list of possible combinations), it would be guessing 🙂

        Kind regards

        Djerro123

        -------------------------------

        If this answered your question, please mark it as the Solution. This also helps others to find what they are looking for.

        Kudo's are welcome 🙂