topic Re: wild card (*) not working on delete activity in Pipelines

wild card (*) not working on delete activity

smeetsh — Wed, 21 Aug 2024 04:36:22 GMT

Hi All,

I create json files through a api, which get overwritten everytime time the pipeline runs. There is a very minute chance that one day a json file may not be overwritten since we have gotten rid of the object and are no longer interested in its history (it will be rare but possible.

I thought ..easy Ill insert a delete activity, but I can't seem to get the wildcard fucntionality to work (I am prolly being a dumbass here). I followed this link from MS and wildcards should work, and I have used them to ingest data in a similar way

https://learn.microsoft.com/en-us/fabric/data-factory/delete-data-activity

The activity step is super simple,

If I take the wildcard out and leave filename blank, it will delete all files in the subfolder, which could work, though I would like to build in some more finesse, incorporating it in a loop and target filenames more selectively.

What am I missing here?

Cheers
Hans

Re: wild card (*) not working on delete activity

Anonymous — Thu, 22 Aug 2024 02:17:56 GMT

Hi smeetsh ,

In data factory, allowed wild cards are *(matches zero or more characters) and ?(matches zero or single character). you can use ^ to escape if your file name has a wildcard or this escape character inside.

Pass output array of filtered file names from Azure functions to ForEach activity to loop each file name and use delete activity inside ForEach activity to dynamically point to the file in the iteration and delete it.

You can use Azure Functions activity to run Azure function in data factory. You can also look at this document: Copy and transform data in Azure Blob Storage - Azure Data Factory & Azure Synapse | Microsoft Learn

Best Regards

Yilong Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Re: wild card (*) not working on delete activity

smeetsh — Thu, 22 Aug 2024 02:22:00 GMT

Thank you for the response, that is exactly what the ms acticle says, but when i try it, it is not what it does. What am I missing ?