Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
4 years ago
Solved

Returning file names only from a big Sharepoint document library

Hi,

 

I need to be able to parse a SharePoint document library and return the file names only and their path.  The fields from the list that I need to return are the Title and FieldValuesAsText.FileRef.  I only need to see files in this list.

 

I am using the Sharepoint Online List connector.

 

When I tested it on a smaller list (10K items), It worked fine (by filtering on the field FileSystemOjbectType =0).

 

The problem I have now is that I need to parse a list that has around 500,000 items.  I ran it (on desktop) for over an hour and it never completed.

 

Does anyone have any clues on a quick and generally less stressful way of doing this?  I am open to using other tools, such as Power Automate to create this dataset and store it elsewhere.

 

Thanks

 

  • "I am using the Sharepoint Online List connector."

     

    That's not the right connector for accessing a Sharepoint document library.  Can you check again?  Maybe post some sanitized Power Query code.

2 Replies

  • "I am using the Sharepoint Online List connector."

     

    That's not the right connector for accessing a Sharepoint document library.  Can you check again?  Maybe post some sanitized Power Query code.

    • Anonymous's avatar
      Anonymous
      Not applicable

      And that is the secret Sauce.

       

      And changed over to use the SharePoint Folder connector and removed the unneeded fields (I.E., I don't need the binary content and I don't need the file attributes).

       

      Once fixed, the process ran (relatively) quickly.

       

      Thanks