Forum Discussion

Sidhant's avatar
Sidhant
Icon for Advocate V rankAdvocate V
1 year ago

Managing and Exporting High-Volume Datasets (~1M Records) in Power BI

Hello everyone, I'm currently working on a use case where I need to manage and export high-volume datasets (~1 million records) using Power BI, and I'm exploring multiple approaches. I’d appreciate ...
Need Help
Sidhant's avatar
Sidhant
Icon for Advocate V rankAdvocate V
11 months ago

Quick update: 
So I had few observations (so I navigated on the web interface for the respective Sharepoint channel)
The files were stored in: Documents/General/ExportLargeData

Previously I had the Base Path as /Shared Documents/General/ExportLargeData

So then I checked from the root level we have:

(All my files are within General/ExportLargeData) and not Shared Documents
Shared Documents also has its own General Folder and currently it creates the folder within it where I have no files

So then I modified the Base path to: 

So now this resolved the 1st half that the folder was created at the right path but again the flow failed at the same action which was added to get the folder metadata that it was not able to get the folder

 

So then when create file action creates the file at the right location and when I had given the same path why did it create inside Shared Documents instead of General

And now when the folder was created at the right path why is not able to get the folder?

Regards,
Sidhant.

Sidhant's avatar
Sidhant
Icon for Advocate V rankAdvocate V
11 months ago

So I figured out the issue: The thing is when we use Create new folder action (SharePoint) therein we specify the List or Library: Documents (which Internally corresponds to Shared Documents) so herein we just need to specify the sub-folder part so then in BaseFolderPath (Compose action) I just specified the same:

This created the folder at the right location and then when we doing the cross -check 'Get Folder metadata using path' I used another Compose action (renamed it to ShareBasePathLocation) wherein I simply specified the complete path (along with SharedDocuments)

So this resolved the folder not found issue, now once the file moves in the respective location we need to break the inheritance (default security) and impose new one so for that I have added the 'Send HTTP request to SharePoint' action

uri:
_api/web/GetFolderByServerRelativeUrl('@{outputs('FolderServerRelativeUrl')}')/ListItemAllFields/breakroleinheritance(copyroleassignments=false,clearSubscopes=true

SiteServerRelativePrefix: (Compose action)
/sites/<SharePointSiteName>

FolderLibraryRelativeFullPath: (Compose action)
@{outputs('Create_new_folder')?['body/{FullPath}']}

FolderServerRelativeURL (Compose action):
Inputs: @{outputs('Create_new_folder')?['body/{FullPath}']}

This is where the flow fails:

"status": 400,
        "message": "{\"odata.error\":{\"code\":\"-1, Microsoft.SharePoint.Client.InvalidClientQueryException\",\"message\":{\"lang\":\"en-US\",\"value\":\"The expression \\\"web/GetFolderByServerRelativeUrl('/sites/<SiteName>/Shared Documents/General/ExportLargeData/<Foldername>')/ListItemAllFields/breakroleinheritance(copyroleassignments=false,clearSubscopes=true\\\" is not valid.\"}}}

Any clue on this v-hashadapu , Gabry , Poojara_D12 

Regards,
Sidhant