Forum Discussion

mumbaicharaja's avatar
mumbaicharaja
Regular Visitor
6 years ago

How to process files from onedrive subfolders recursively

I want to know how to recursively process files in subfolders of onedrive. This is needed for consolidating the data from all the excel files in given folder and all subfolders underneath.. I already have PowerQuery written to go to a specific folder and process all the files in that. However, it needs to happen recursively for all the sub-folders and sub-sub-folder in that. Any guidance please?

Need Help

5 Replies

  • edhans's avatar
    edhans
    Icon for Community Champion rankCommunity Champion
    6 years ago

    Do this using the SharePoint connector mumbaicharaja 

    1. Go to OneDrive in your browser. You will get a URL that looks like this:
      1. https://tenantname-my.sharepoint.com/personal/username_tenantname_com/_layouts/15/onedrive.aspx
    2. You want to use this part - https://tenantname-my.sharepoint.com/personal/username_tenantname_com
    3. Sign in
    4. Click Transform Data
    5. Go to the folder path column
    6. Filter for the folder you want. You want to use "Contains" so it will get that folder and all folders with that info below it.
    7. Filter by file type is needed. A good idea just in case other files creep in there.
    8. Press the Combine button and follow the wizard 
       
       
       

  • Anonymous's avatar
    Anonymous
    Not applicable
    6 years ago

    it is just a guess, but try if this is a good starting point:

     

     

    let

   dirall=(path)=>
   let 
    Source = Folder.Contents(path), 
    dir=List.Accumulate(Table.ToRows(Source[[Content],[Name]]),{},(s,c)=>if Value.Is(c{0}, type table) then s&@dirall(path &"\"& c{1})  else s&{c{1}} )
    in dir

in
    dirall

     

     

    this function recursively goes through all the folders and subfolders of a directory and supplies only the name of the files.
    but only you know what else to do with these files

    • Anonymous's avatar
      Anonymous
      Not applicable
      6 years ago

      here a version that add some action to the files.

       

      let

   dirall=(path)=>
   let 
    Source = Folder.Contents(path), 
    dir=List.Accumulate(Table.ToRows(Source[[Content],[Name]]),{},(s,c)=>if Value.Is(c{0}, type table) then s&@dirall(path &"\"& c{1})  else s&{[name=c{1},len=Text.Length(Text.BeforeDelimiter(c{1},".")) ]} )
    in dir

in
    dirall

       

       

      for each files name the last number is the number file in the subfolder.

       

      f212 for example is the second file in the filder f21 which is the first subfolder of folder  f2 wich is the second subfolder of folder "test"

       

       

       

       

       

       

       

      • Anonymous's avatar
        Anonymous
        Not applicable
        6 years ago

        may be this could be more usefull to your goal

         

         

         

         

        let

   dirall=(path)=>
   let 
    Source = Folder.Contents(path), 
    dir=List.Accumulate(Table.ToRows(Source[[Content],[Name]]),{},(s,c)=>if Value.Is(c{0}, type table) then s&@dirall(path &"\"& c{1})  else s&{path & "\"& c{1}} )
    in  
dir
in
    dirall

         

         

         

         

        having all the files with complete path you can apply your function (I suppose)

         

        #####

        in the following form it is probably easier to understand

         

         

         

         

        let
  dirall = (path) => List.Accumulate(
      Table.ToRecords(Folder.Contents(path)), 
      {}, 
      (s, c) => 
        if Value.Is(c[Content], type table) then 
          s & @dirall(path & "\" & c[Name])
        else 
          s & {path & "\" & c[Name]}
    )
in
  dirall