apis and embedding

2 Topics

mssparkutils.notebook.run not running notebooks in isolated session context
When calling multiple notebooks using the mssparkutils, they do not run in thread-safe manner and will share session context veriables. This is true if you call two notebooks sequentially using mssparkutils.notebook.run sequentally in a notebook or if you call in parallel using mssparkutils.notebook.runMultiple(). In either case, values set in session using spark.conf.set will be visible by other other notebooks. And if running in parallel, the notebooks may interfere with each other if using the same variable names. Temporary Views are also shared across their sessions, which could also create issues. DBUtils runs notebooks in isoluation and I believe mssparkutils should also run in isolation.
Solved
shietpas_cbts
2 years ago Place Fabric platform
3.4KViews
0likes
4Comments
Create database for default is not permitted using Apache Spark in Microsoft Fabric.
Receiving the error in the title when trying to create a delta table programmatically via notebook. I've created a new Lakehouse, created a new PL, and tried debugging using a very simple manually created dataframe. I've also tried in two different tenants, EUS and CUS. Problem has persisted for about a week. Any ideas?
WCrayger
3 years ago Place Fabric platform
1.4KViews
0likes
2Comments