Forum Discussion
How to use variable library for deployment from Test to Prod?
I am working on the topic of deployment from Test to Prod systems and I have read that the variable library can be helpful.
Let’s imagine a very simple case:
I have a Dataflow Gen2 that reads data (a simple table with one column and one row) and writes it into a Lakehouse in the Test system.
My questions are:
Which variables do I need? (please list all required variables)
What is the best way to perform the deployment from Test to Prod?
Thank you in advance!
Hi RouyAlfahel,
Dataflow Gen2 can be made dynamic by referencing Fabric Variable Library in the Power Query M code, which allows the source to switch between Test and Prod environments without relying on a Copy Data activity. However, the destination (sink) remains fixed at design time and cannot be parameterized, so adjustments for the sink must still be handled manually or through pipeline configuration.in humanize way
Use Fabric variable libraries in Dataflow Gen2 (Preview) - Microsoft Fabric | Microsoft Learn
Thank you.
6 Replies
- AntoineW
Super User
Hello RouyAlfahel,
You’re right, you can use the Variable Library to parameterize both the source connection string and the destination when working with Pipelines or Notebooks. This is the recommended approach if you want to fully automate deployment from Dev → Test → Prod.
For Dataflow Gen2, there are still some limitations:
You can parameterize the source (server, database, schema, table, etc.) directly in Power Query M using variables.
But the destination (sink) can’t be parameterized with variables — it must be defined at design time and adjusted manually after deployment.
Docs:
- Variable library integration with Dataflow Gen2
- https://learn.microsoft.com/en-us/fabric/cicd/variable-library/variable-library-overview
- https://learn.microsoft.com/en-us/fabric/cicd/variable-library/tutorial-variable-library
Hope it can help you !
Best regards,
Antoine
- v-saisrao-msft
Community Support
HI RouyAlfahel,
Have you had a chance to review the solution we shared by AntoineW ? If the issue persists, feel free to reply so we can help further.
Thak you.
- v-saisrao-msft
Community Support
Hi RouyAlfahel,
Checking in to see if your issue has been resolved. let us know if you still need any assistance.
Thank you.
- RouyAlfahelFrequent Visitor
I created a simple table and stored it in a Lakehouse using Dataflow Gen2. Then, I used this Lakehouse as a data source. After defining my variables in the Variable Library, I configured the Pipeline dynamically so that it uses a dynamic source and destination address — and it works successfully.
However, I would like to know whether it’s possible to add this logic directly in the query code (M language), so that I wouldn’t need to use a Copy Data activity in the pipeline during deployment.
Thank you very much for your support!
- v-saisrao-msft
Community Support
Hi RouyAlfahel,
Dataflow Gen2 can be made dynamic by referencing Fabric Variable Library in the Power Query M code, which allows the source to switch between Test and Prod environments without relying on a Copy Data activity. However, the destination (sink) remains fixed at design time and cannot be parameterized, so adjustments for the sink must still be handled manually or through pipeline configuration.in humanize way
Use Fabric variable libraries in Dataflow Gen2 (Preview) - Microsoft Fabric | Microsoft Learn
Thank you.
- Theo_MS
Advocate I
Hey,
It could be great that Dataflow Gen2 destination have the same implementation as Copy Job.
It seems the best way to use.