Forum Discussion

4 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi Zafar06 ,

    Glad to help again, I see you applied to ForEach.

    So with those limitations in mind, I think you might want to take a look at this documentation, it might be able to help you: Data Pipeline Performance Improvements Part 2: Creating an Array of JSONs | Microsoft Fabric Blog | Microsoft Fabric

     

     

     

    Best Regards

    Yilong Zhou

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

  • frithjof_v's avatar
    frithjof_v
    Community Champion

    Zafar06 

     

    I don't think that the array variable is empty, even if the output message is empty like this {}.

     

    It seems like values like "2023-07-09" are being appended to your variable.

     

    So I think your variable called 'days' actually contains an array of dates (each date formatted as strings like "xxxx-xx-xx").

     

    Are you having issues utilizing that variable in a subsequent step?

    • Zafar06's avatar
      Zafar06
      Frequent Visitor

      yes, variables are not working. I am going to use each variable in API URL link ( example: relative url...general_ledger_accounting_date=Variable). Want retrieve data per day and append it. Otherwise data is so huge, it can not be loaded. 

      • frithjof_v's avatar
        frithjof_v
        Community Champion

        Could you have a look in the pipeline run history and check if one copy activity gets executed for each date in your date range? 

        Do you get any error messages?

         

         

        Can you show what code you are using inside all of your activities? 

         

        (Please hide any sensitive information)

         

        Can you also show which variables you have in your pipeline and what type of variable they are (array, string, integer, etc.).

         

        In general I think your approach should work but perhaps something which needs to be changed in the code.

         

         

         

        One thing you can do if you want to verify the contents of the array variable ('myArrayVariable'), is to add a new string variable 'myStringVariable' and then in the step after the ForEach where you append values to your array variable ('myArrayVariable'), you can "print" the array variable to the string variable 'myStringVariable' by using set variable 'myStringVariable' and this expression: 

         

        @join(variables('myArrayVariable'), ',')

         

         

        After the pipeline has run, you can inspect the output of the Set variable step for 'myStringVariable'.

         

        In this example screenshot, I have used the names DummyDateArray ('myArrayVariable') and JoinDummyDateArray ('myStringVariable').

         

         

        I think this is an easy way to inspect the contents of the variable.

         

        I'm not so experienced with data factory, and I don't know if there are some easier ways to inspect the contents of an array variable after a ForEach activity.