Forum Discussion
Filter in $project cursor in copy activity using Atlas MongoDB connector doesn't work
Hi pafnuty ,
I have a suggestion that you can create an Azure Function to handle complex filtering logic. This Azure Function is then called in the data factory to process the data and pass the results to the next copy activity.
Best Regards,
Adamk KongIf this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Hi pafnuty ,
Ensure that the MongoDB Atlas connector configured in the replication activity supports the use of $filter in the $project phase.
You can refer to the following official documentation:
Copy data from or to MongoDB Atlas - Azure Data Factory & Azure Synapse | Microsoft Learn
Best Regards,
Adamk Kong
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Thanks Anonymous
Seems like this is not supported by Azure Data Factory
Do you think there is some other way of applying this filtering with the help of Fabric?
I have really a lot of data in case I'm not using filtering, which increase processing time in tens times
Hi pafnuty ,
I have a suggestion that you can create an Azure Function to handle complex filtering logic. This Azure Function is then called in the data factory to process the data and pass the results to the next copy activity.
Best Regards,
Adamk KongIf this post helps, then please consider Accept it as the solution to help the other members find it more quickly.