Forum Discussion
How to do nested for loops in data factory
I'm pretty new to data factory, and I'm creating a pipeline where a nested for loop is required. As far as I can notice nested loops are not allowed, so I'm open to listen for alternatives to get this solved.
I'm making an API request where there are two things I need to iterate. One is a date, the other one the authorization key. Each authorization key represents a different data set I need to have access. Regarding the dates I just need to iterate for every single day in the month.
So basically, I will need to make a request to the this API for every single day (not a range due to the limit of items retrieved) and for every Bearer code (that will represent each dataset I have to request).
So far I've done it for a single Bearer, but then I will need to re run all the pipeline for a different Bearer making it very time consuming. That's why one solution is to do it with a nested loop.
An alternative I thought of is creating a ForEach instance for one of the variables, while fo the other is SetVariable with array values, and make an iteration for each element inside that variable. The issue is that I quite don't know how to get this done properly.
Any help will be much appreciate it.
Hi Anonymous ,
Parent Pipeline:
Child Pipeline -
Note: Make sure to add pipeline level parameters at child Pipeline
Please check foreach iterator and parameter in pipeline expression builder.
On execution you will get the required output.
Hope this is helpful. Please let me know incase of further queries.
6 Replies
Hi Anonymous ,
Thanks for using Fabric Community.
Nested foreach activity is not allowed. But you could use an execute pipeline activity inside the foreach activity. And in the nested pipeline, you could have another foreach.
For Example:
Parent Pipeline
Child Pipeline
Above example just helps you in getting an idea, on how to use create nested loop in fabric data factory.
If you can provide some more inputs then we can build a pipeline along with nested loop that can execute your task.
Hope this is helpful. Please let me know incase of further queries.Thank you for your answer Anonymous
I tried to replicate your answer and created two pipelines with a for loop in each. The parent pipeline will iterate over Fruits
And the child pipeline over Serving types
The thing is I don't understand how to incorporate the array values of the Fruits inside Serving types.
What I'm expecting is to retrieve variable values of 'Apple Sliced', 'Apple Smashed', 'Apple Shake', 'Orange Sliced', 'Orange Smashed', 'Orange Shake', and so on.
This ain't my actual data, but I'm trying to understand the logic behind it and apply it to my pipeline that, in general terms, will be something like since I'm iterating over strings.
Hi Anonymous ,
Parent Pipeline:
Child Pipeline -
Note: Make sure to add pipeline level parameters at child Pipeline
Please check foreach iterator and parameter in pipeline expression builder.
On execution you will get the required output.
Hope this is helpful. Please let me know incase of further queries.