Looping to copy tables using a dataflow gen2

[pmscorca]

Hi,

I'd like to know if it is possible to implement a loop using a dataflow in order to copy some tables from a lakehouse to another lakehouse or warehouse.

Any suggests to me, please? Thanks

[NandanHegde · Accepted solution]

Unfortunately, based on my understanding as of now dataflows cannot be parameterized for the source and sink names. So you cannot have a meta data driven framework and thus there is no sense with looping the dataflows

[v-cboorla-msft]

Hi @pmscorca 

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
In case if you have any resolution please do share that same with the community as it can be helpful to others.
Otherwise, will respond back with the more details and we will try to help.

Thank you.

[frithjof_v]

I never tried, but I don't think looping in order to create multiple destination tables is supported in Dataflow.

What is the reason you need to loop and create copy tables?

Why not use e.g. shortcuts or cross-database queries instead?

I would guess Notebook for looping.

Maybe it's also possible in Data pipeline?