Load list of table name in pipeline with foreach

[Theo_MS]

Newbie here!

I would like to establish an action that I couldn't find a way or answer.

The purpose is to load a list of specific table names from one source of data.

This list is subject to change (new/modify/delete values).

Example, list of tables:

• Table1
• Table3
• Table7

What is the best approch?

Regards,

Théo

[nilendraFabric · Accepted solution]

Hello @Theo_MS 

To dynamically load a list of tables in Microsoft Fabric pipelines with changing table names, use a metadata-driven approach with Lookup and ForEach activities

1 Retrieve Table Names with a Lookup Activity

Use a query or stored procedure to fetch the latest table list from your source system:

Store your table names in a configuration table for easier updates

Ensure the Lookup returns results as an array (e.g., `@activity('Lookup1').output.value`)

Iterate with ForEach Activity
Configure the ForEach activity to process each table

Iterate with ForEach Activity
Configure the ForEach activity to process each table
"items": "@activity('Lookup1').output.value"

1. Lookup Activity (Get Table List)
2. ForEach Activity (Iterate Tables)
 If Activity (Filter Table)
Copy Data Activity (Valid Tables)
Optional: Logging Activity (Skipped Tables)

https://github.com/Azure/AnalyticsinaBox/tree/main/Fabric/src/deployment/Pattern1and2

if this is helpful please accept the answer 

[Theo_MS]

Hi @nilendraFabric,

thanks for you explanation and steps.

Currently in pipeline, it seems impossible to use a stored procedure created in a Lakehouse.

The "stored procedure" activity accepts only Warehouse.

How can I store table names in a configuration in a Lakehouse?

Do you have different proposal ?

Regards,

Théo

[nilendraFabric]

Add a Lookup activity to your pipeline:
• Source: Lakehouse table `config_table`.
• Configure to return all rows (not just the first)