Get Data from Azure Storage Table

[FotFly]

Hi all,

I want to get some data from an azure storage table. Since my dataset will be large I tried to reduced it filtering out some rows. When I hit close and apply around 6 hours later I get a timeout error. I tried different configuration of filtering to reduce the size but each time it seems that it loads around 6GB data from the storage which takes too long and that is why I get the timeout.
Does anyone know whether power bi first loads the data and then performs the transformation or the query is executed on the source and then I get the remaining data for this specific source? I know that for sql tables the second way applies but since storage tables are NoSql is there a different treatment?

Thanks in advance.

[Anonymous]

Hi @FotFly ,

Whether the advice given by lbendlin has solved your confusion, if the problem has been solved you can mark the reply for the standard answer to help the other members find it more quickly. If not, please point it out.

Best Regards,

Neeko Tang

[lbendlin]

Does anyone know whether power bi first loads the data and then performs the transformation or the query is executed on the source and then I get the remaining data for this specific source?

That depends on the transforms you do.  Have you heard about "Query Folding"  yet?

Queries from Power BI Desktop do not usually time out. Your timeout must come from the Azure Storage source.

Consider using incremental refresh.  

[FotFly]

Hi, thank you for your answer. I guess my question should be better phrased as does power query support query folding for Azure storage table. In my tests does not seem to do so (even by applying basic transformations). I was hoping if someone could have earlier experience on that which could help.

[lbendlin · Accepted solution]

In my experience query folding is limited to data sources that expose some sort of SQL interface (including Oracle, and even SFDC).  Yours sounds more like a blob storage and would have to be ingested as is before applying the transforms.