Forum Discussion
Parameterization of environment when running notebook activity
- 1 year ago
I think I finally found an API that would work. This could be used from an ADF pipeline. It appears that it may allow us to specify what environment to use.
https://learn.microsoft.com/en-us/fabric/data-engineering/notebook-public-api#run-a-notebook-on-demand
Hi Anonymous
In Fabric-spark there is a concept called the "environment" of the notebook. I'm not using the word environment in a generalized sense. I'm referring to the selection of this notebook "environment".
I think I understand your dynamic reconfiguration of the notebook spark session, but that isn't what I'm looking for...
The main thing I need to be able to do is swap back and forth between "custom" spark pool and the "starter" spark pool. This would allow me to isolate bugs in the platform which are unusual in that they impact one type of spark pool and not the other. In the perfect world the "custom" pools wouldn't have additional reliability concerns. But we are facing issues, and Microsoft does not seem eager to engage with their "professional" support cases. So I am doing additional work to troubleshoot error messages for them. My plan is currently to pinpoint the bug, leverage this bug to create a painful week-long outage in production, and then open a case with "unified" support (it is Microsoft's secondary support organization, which is presumably more willing to work on bugs in Fabric). I've learned that you sometimes have to pay a bit more to get an Azure bug fixed (or even get it to be acknowledged).
Hi dbeavon3
Thank you for your feedback, we understand your frustration about this matter.
Based on the detailed information you provided, I realize that this aspect is not the focus of this forum. You can go to the following link to ask questions, where more professional people will assist you in solving the problem:
Get Help with Data Warehouse - Microsoft Fabric Community
Thank you again for your understanding. If you have more questions about Dataflow or Data Pipeline, you are welcome to continue asking in this forum! We'll do our best to help.
Regards,
Nono Chen
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
- dbeavon31 year ago
Memorable Member
I think I finally found an API that would work. This could be used from an ADF pipeline. It appears that it may allow us to specify what environment to use.
https://learn.microsoft.com/en-us/fabric/data-engineering/notebook-public-api#run-a-notebook-on-demand- Anonymous1 year agoNot applicable