topic Re: Dynamic SQL Connection (ForEach Loop) in Data Engineering

Dynamic SQL Connection (ForEach Loop)

Tech122 — Tue, 18 Mar 2025 13:02:58 GMT

Hi Everyone,

I'm trying to loop through multiple databases, each with a different connection string. All the databases have the same structure and format. My goal is to parameterize the connection and query so that I can extract information from each database dynamically.

So far, my findings suggest that this is not yet supported in Fabric. Any updates on this would be greatly appreciated.

Re: Dynamic SQL Connection (ForEach Loop)

NandanHegde — Wed, 19 Mar 2025 06:06:14 GMT

Unfortunately, based on my understanding it is not possible to parameterzie connections for components other than fabric components.

So your guess is correct

Re: Dynamic SQL Connection (ForEach Loop)

v-pnaroju-msft — Wed, 19 Mar 2025 06:41:53 GMT

Thank you, @NandanHegde , for your response.

Hi @Tech122,

We sincerely appreciate your inquiry on the Microsoft Fabric Community Forum.

In addition to the response provided by @NandanHegde, each connection in Fabric is predefined, and dynamically modifying the connection string within a pipeline or dataflow is not possible.

We kindly request you to stay updated by referring to the following documentation:
What's New? - Microsoft Fabric | Microsoft Learn

If you find our response helpful, we request you to mark it as the accepted solution and provide kudos, as this will assist other community members facing similar queries.

Thank you.

Re: Dynamic SQL Connection (ForEach Loop)

NandanHegde — Wed, 19 Mar 2025 06:45:12 GMT

Both :

1) Making the connection value dynamic (parameterzing the connectin values)

2) And dynamically updating the connectionid in the activity for non fabric objects

are not possible as of today.

Re: Dynamic SQL Connection (ForEach Loop)

Tech122 — Wed, 19 Mar 2025 09:30:01 GMT

Thank you for all the responses. I was wondering if there is any workaround or alternative approach to achieve dynamic connection handling in this scenario. Any suggestions or best practices would be greatly appreciated.

Thank you again for your assistance.

Re: Dynamic SQL Connection (ForEach Loop)

Tech122 — Wed, 19 Mar 2025 09:33:12 GMT

Thank you for the the responses. I was wondering if there is any workaround or alternative approach to achieve dynamic connection handling in this scenario. Any suggestions or best practices would be greatly appreciated.

Thank you again for your assistance

Re: Dynamic SQL Connection (ForEach Loop)

Tech122 — Wed, 19 Mar 2025 09:38:29 GMT

Thank you for the responses. Is there any workaround or alternative approach to achieve the Dynamic SQL Connection handling in this scenario? Any suggestions or best practices would be greatly appreciated.

Re: Dynamic SQL Connection (ForEach Loop)

v-pnaroju-msft — Thu, 20 Mar 2025 16:41:44 GMT

Hi Tech122,

Please find below some possible workarounds that may help resolve the issue:

If feasible, consider migrating your data to Fabric-native components such as Lakehouse, Warehouse, or KQL databases. These components support dynamic content within pipelines and allow for the parameterization of certain aspects.
If possible, consolidate data from multiple databases into a single data lake. This will enable you to query a centralised location, simplifying connection management and potentially enhancing performance.

If you find our response helpful, we request you to mark it as the accepted solution and provide kudos, as this will assist other community members facing similar queries.

Thank you.

Re: Dynamic SQL Connection (ForEach Loop)

v-pnaroju-msft — Sun, 23 Mar 2025 18:19:31 GMT

Hi Tech122,

We have not received a response from you regarding the query and were following up to check if you have found a resolution. If you have identified a solution, we kindly request you to share it with the community, as it may be helpful to others facing a similar issue.

If you find the response helpful, please mark it as the accepted solution and provide kudos, as this will help other members with similar queries.

Thank you.

Re: Dynamic SQL Connection (ForEach Loop)

Tech122 — Mon, 24 Mar 2025 11:01:33 GMT

Thank you @v-pnaroju-msft , @NandanHegde and all. It appears that there is still a limitation in the Microsoft Fabric environment.