Forum Discussion

Anagoor's avatar
Anagoor
New Member
10 months ago
Solved

How to parameterize output from Dataflow

Good morning, I'm having an issue parametrizing the output of a Dataflow gen2.   In particular given some parameters I'm basically choosing the table to create, those tables have in general differ...
Dataflow
  • AntoineW's avatar
    AntoineW
    10 months ago

    Hello Anagoor,

     

    It's a bit frustrating, I'm also experiencing that in Dataflow Gen2, the destination table (sink) can’t be parameterized, because it isn’t managed through Power Query M. You can only parameterize the source side.

    Your failures come from trying to use one dataflow output for multiple tables with different schemas — schema mismatches will always break the run.

    The recommended approaches are:

    • Use a Pipeline with a Notebook or Copy activity for dynamic routing and schema-aware loads, or

    • Create separate Dataflows Gen2 for each target schema.

     

    Hope it can help you ! 

    Best regards,

    Antoine

AntoineW's avatar
AntoineW
Super User
10 months ago

Hello Anagoor,

 

It's a bit frustrating, I'm also experiencing that in Dataflow Gen2, the destination table (sink) can’t be parameterized, because it isn’t managed through Power Query M. You can only parameterize the source side.

Your failures come from trying to use one dataflow output for multiple tables with different schemas — schema mismatches will always break the run.

The recommended approaches are:

  • Use a Pipeline with a Notebook or Copy activity for dynamic routing and schema-aware loads, or

  • Create separate Dataflows Gen2 for each target schema.

 

Hope it can help you ! 

Best regards,

Antoine