Forum Discussion

AnibalBenedetto's avatar
AnibalBenedetto
Frequent Visitor
2 years ago

Error using Dataflow activity in Pipeline with dynamic content

Hello!, seems to be an issue with pipelines. When you add a Dataflow “activity” and set it with dynamic content (introducing his Dataflow object ID) you can Validate, Save and Run correctly, but if you close it, you won’t be able to open again.

 

 

 

I am trying to build a pipeline to run a list of Dataflows in a ForEech Activity.

Thanks!

 

data engineering

14 Replies

  • AndyDDC's avatar
    AndyDDC
    Most Valuable Professional
    2 years ago

    Hi AnibalBenedetto are you having issues opening the pipeline itself after you closed it?  I tried explicitly stating the dataflow id and also using a variable, and both seemed to validate and run fine. I could close the pipeline and re-open.

     

    Have you tried using a variable and seeing if that makes a difference?

     

    Also, what does the FSON View look like? Click View > View JSON Code (after you've got the pipeline working again) just to see what the JSON looks like

     

     

     

     

     

    • AnibalBenedetto's avatar
      AnibalBenedetto
      Frequent Visitor
      2 years ago

      Thanks for your time!
      Yes, I can’t open the pipeline artifact after close it.
      I don’t know way, but I can’t reproduce your recommendations using the ID value or a Variable.
      The JSON Code is empty in my case and I can’t edit / recover the original pipeline.

      • AndyDDC's avatar
        AndyDDC
        Most Valuable Professional
        2 years ago

        Can you try recreating the pipeline and using the variable?

  • v-cboorla-msft's avatar
    v-cboorla-msft
    Microsoft Employee
    2 years ago

    Hi AnibalBenedetto 

     

    Thanks for using Microsoft Fabric Community.

    I tried to repro the scenario and got the same error when giving the "Dataflow Object ID" dynamically.

    Instead of hard coding the value we can give the input as a parameter as shown in the screenshots below:

    You can open the pipeline now even after closing it without any error.

     

    Hope this helps. Please do let us know if you any further queries.

     

     

     

  • DiegoPLAB's avatar
    DiegoPLAB
    Advocate I
    2 years ago

    Hi everybody,

    I have the same issue on a pipeline that invokes a dataflow with dynamic content.

    However, in my case the dataflow is inside a ForEach iterator and I was trying to invoke a dataflow by its name building an expression like this:

    @concat('Clean_', item())

    The result of the expression is correct: dataflows with the expected names exist in my workspace. However, when I ran the pipeline I got an error saying that dataflows doesn't exist.

    Now I can't open anymore the pipeline.

     

    It would be nice if you can dynamically invoke a dataflow using its name rather than the internal ID.

     

    Thanks.