Enable Dataflow Gen2 to Dynamically Adjust Schema Mapping & Resource Paths Using Pipeline Parameters

[ChunPo]

Please consider enabling Dataflow Gen2 to:

1. Dynamically adjust schema mapping at runtime based on the resolved parameter value.
2. Allow resource paths (such as Lakehouse table names) to be fully driven by public parameters from pipelines.
3. Support end‑to‑end dynamic data ingestion patterns commonly used in modern data engineering.

@https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-parameters#considerations-and-limitations

[miguel]

Hi! 

Could you tell us a bit more about the scenario for #1 and #3?

For #2, and specifically for your case with Lakehouse destinations, changing the name of your table is completely supported for Dataflow public parameters. You can follow the tutorial below for more information on how to make that happen:

Parameterized Dataflow Gen2 - Microsoft Fabric | Microsoft Learn

[ChunPo]

I have contacted the support team and here is their response: Please review the document below, which outlines that using public parameters is limited when parameters change the resource path for either the source or destination. Additionally, parameters cannot be used to modify schema mapping. @https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-parameters#considerations-and-limitations

[miguel]

That is correct information. However, changing the LakehouseId, WorkspaceId or TableName when using the Lakehouse connector (point #2) do not impact the resource path. By resource path we are referring to the connection component as described in the Dataflow item definition from the link below:

Dataflow definition - Microsoft Fabric REST APIs | Microsoft Learn

If you could tell us a bit more about your scenario for #1 and #3, perhaps with an example, that would help tremendously.