Query Azure Storage Tables

[daniel_sql_27]

I'm using Fabric Data Pipelines to bring over data from Azure Storage Tables, but I need to filter the data to get the latest data only. I'm trying to write a query to get data where the timestamp column is in the last 5 days. I can write a query like Timestamp ge '12/25/2024' and it will return results, but I cannot make it work with dynamic content. I would like something like Timestamp ge @addDays(utcNOW(),-5); but when I try to do that it gives me errors about invalid inputs. Any one have success dynamically query azure storage tables? 

[daniel_sql_27 · Accepted solution]

I found this page. I wasn't able to get it to work with the Timestamp field, but I have another field stored as a string called ProcessedAt and that worked:

https://stackoverflow.com/questions/57176460/table-storage-dynamics-query-in-data-factory

My query looks like this:

Variable Now = @addDays(formatDateTime(utcNow(),'MM-dd-yyyy'),-5)

Query = @concat('ProcessedAt ge ''', variables('Now'),'''')

Not very intuitive, but at least it works. 

[Anonymous]

Hi @daniel_sql_27 ,

I'm glad you were able to solve the problem yourself, and I'm sure it will help other users on the forum!

Best Regards

Yilong Zhou