Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Get certified in Microsoft Fabric—for free! For a limited time, the Microsoft Fabric Community team will be offering free DP-600 exam vouchers. Prepare now

Reply
PauloAlvarez
Frequent Visitor

How to do nested for loops in data factory

I'm pretty new to data factory, and I'm creating a pipeline where a nested for loop is required. As far as I can notice nested loops are not allowed, so I'm open to listen for alternatives to get this solved.

 

I'm making an API request where there are two things I need to iterate. One is a date, the other one the authorization key. Each authorization key represents a different data set I need to have access. Regarding the dates I just need to iterate for every single day in the month.

 

So basically, I will need to make a request to the this API for every single day (not a range due to the limit of items retrieved) and for every Bearer code (that will represent each dataset I have to request).

 

So far I've done it for a single Bearer, but then I will need to re run all the pipeline for a different Bearer making it very time consuming. That's why one solution is to do it with a nested loop.

 

An alternative I thought of is creating a ForEach instance for one of the variables, while fo the other is SetVariable with array values, and make an iteration for each element inside that variable. The issue is that I quite don't know how to get this done properly.

 

Any help will be much appreciate it. 

4 ACCEPTED SOLUTIONS
Anonymous
Not applicable

Hi @PauloAlvarez ,

Parent Pipeline:

vgchennamsft_0-1711113366065.png

 

vgchennamsft_1-1711113397830.png


Child Pipeline -
Note: Make sure to add pipeline level parameters at child Pipeline 

vgchennamsft_2-1711113504292.png

Please check foreach iterator and parameter in pipeline expression builder.

vgchennamsft_3-1711113556002.png

On execution you will get the required output.

Hope this is helpful. Please let me know incase of further queries.

 

View solution in original post

Thank you very much, I think I get now the idea and going to implement it. If any further questions arise I will give you a touch. As of now, this solution seems to do the trick. Thank you @Anonymous !

View solution in original post

Anonymous
Not applicable

Hi @PauloAlvarez ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered.
Otherwise, will respond back with the more details and we will try to help .

View solution in original post

Anonymous
Not applicable

Hi @PauloAlvarez ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered.
Otherwise, will respond back with the more details and we will try to help .

View solution in original post

6 REPLIES 6
Anonymous
Not applicable

Hi @PauloAlvarez ,

Thanks for using Fabric Community.
Nested foreach activity is not allowed. But you could use an execute pipeline activity inside the foreach activity. And in the nested pipeline, you could have another foreach.

For Example:
Parent Pipeline

vgchennamsft_0-1711104700216.png


Child Pipeline

vgchennamsft_1-1711104735125.png


Above example just helps you in getting an idea, on how to use create nested loop in fabric data factory.
If you can provide some more inputs then we can build a pipeline along with nested loop that can execute your task.

Hope this is helpful. Please let me know incase of further queries.

Thank you for your answer @Anonymous 

I tried to replicate your answer and created two pipelines with a for loop in each. The parent pipeline will iterate over Fruits 

PauloAlvarez_0-1711108998456.png

And the child pipeline over Serving types

PauloAlvarez_1-1711109024415.png

The thing is I don't understand how to incorporate the array values of the Fruits inside Serving types.

 

What I'm expecting is to retrieve variable values of 'Apple Sliced', 'Apple Smashed', 'Apple Shake', 'Orange Sliced', 'Orange Smashed', 'Orange Shake', and so on.

 

This ain't my actual data, but I'm trying to understand the logic behind it and apply it to my pipeline that, in general terms, will be something like since I'm iterating over strings.

Anonymous
Not applicable

Hi @PauloAlvarez ,

Parent Pipeline:

vgchennamsft_0-1711113366065.png

 

vgchennamsft_1-1711113397830.png


Child Pipeline -
Note: Make sure to add pipeline level parameters at child Pipeline 

vgchennamsft_2-1711113504292.png

Please check foreach iterator and parameter in pipeline expression builder.

vgchennamsft_3-1711113556002.png

On execution you will get the required output.

Hope this is helpful. Please let me know incase of further queries.

 

Thank you very much, I think I get now the idea and going to implement it. If any further questions arise I will give you a touch. As of now, this solution seems to do the trick. Thank you @Anonymous !

Anonymous
Not applicable

Hi @PauloAlvarez ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered.
Otherwise, will respond back with the more details and we will try to help .

Anonymous
Not applicable

Hi @PauloAlvarez ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered.
Otherwise, will respond back with the more details and we will try to help .

Helpful resources

Announcements
September Hackathon Carousel

Microsoft Fabric & AI Learning Hackathon

Learn from experts, get hands-on experience, and win awesome prizes.

October NL Carousel

Fabric Community Update - October 2024

Find out what's new and trending in the Fabric Community.