Forum Discussion

AlejoSQL's avatar
AlejoSQL
Advocate I
9 years ago

Choosing a specific folder as source down in Azure Blob storage folder hierarchy

HI all:

 

One question : I am trying to read json files from a particular folder (9 levels deep) in Azure Blob storage folder. -In other words, it is not a root folder-

 

I tried

 

1) Fiddling with

Source = AzureStorage.Blobs("https://storageaccount.blob.core.windows.net:443"),
#"alldata" = Source{[Name="rootfolderinblobstorage/folder1/folder2/Desiredfolder"]}[Data],

this threw an error stating that the path was not representing a valid azure resource ...

 

2) so I tried "ok, get everything -inefficient- , and filter locally"

 

#"Filtered Rows" = Table.SelectRows(#"alldata", each Text.StartsWith([Name], "/folder1/folder2/Desiredfolder"))
in
#"Filtered Rows"

 

but this threw an "access to the resource is forbidden" message and asked me for the token again (providing it does nothing and the message persists)

 

Any ideas anyone? Am I missing something totally obvious?

 

Thank you all!

 

Alejandro Leguizamo

 

Need Help

2 Replies

  • mike_honey's avatar
    mike_honey
    Memorable Member
    9 years ago

    I haven't done much with Azure Blob storage specifically, but I've seen similar issues with other sources (even SQL). My default hack is to add a Table.Buffer step, e.g. after your alldata step:

     

    Table.Buffer ( #"alldata" )

     

    From my understanding that forces PQ to stop "Query Folding" (interpreting downstream steps and sending their logic up to the data source ).