Forum Discussion
Query Azure Storage Tables
I'm using Fabric Data Pipelines to bring over data from Azure Storage Tables, but I need to filter the data to get the latest data only. I'm trying to write a query to get data where the timestamp column is in the last 5 days. I can write a query like Timestamp ge '12/25/2024' and it will return results, but I cannot make it work with dynamic content. I would like something like Timestamp ge @addDays(utcNOW(),-5); but when I try to do that it gives me errors about invalid inputs. Any one have success dynamically query azure storage tables?
I found this page. I wasn't able to get it to work with the Timestamp field, but I have another field stored as a string called ProcessedAt and that worked:
https://stackoverflow.com/questions/57176460/table-storage-dynamics-query-in-data-factory
My query looks like this:Variable Now = @addDays(formatDateTime(utcNow(),'MM-dd-yyyy'),-5)Query = @concat('ProcessedAt ge ''', variables('Now'),'''')
Not very intuitive, but at least it works.
2 Replies
I found this page. I wasn't able to get it to work with the Timestamp field, but I have another field stored as a string called ProcessedAt and that worked:
https://stackoverflow.com/questions/57176460/table-storage-dynamics-query-in-data-factory
My query looks like this:Variable Now = @addDays(formatDateTime(utcNow(),'MM-dd-yyyy'),-5)Query = @concat('ProcessedAt ge ''', variables('Now'),'''')
Not very intuitive, but at least it works.Hi daniel_sql_27 ,
I'm glad you were able to solve the problem yourself, and I'm sure it will help other users on the forum!
Best Regards
Yilong Zhou
