Forum Discussion
Post PatronHigh Currency for Notebooks
Hi tan_thiamhuat ,
Yes, writing to a persistent table with saveAsTable makes the data accessible to any notebook connected to the same lakehouse and Spark environment, including both high concurrency and standard sessions. This is useful for sharing data broadly across sessions.
However, temporary views are limited to the same Spark session, so to share them between notebooks, those notebooks must join the same high concurrency session. If you need quick, session-scoped sharing, use temporary views with shared sessions; for wider access across different sessions, persistent tables are the best option.
If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.
Thank you.
Super UserHi tan_thiamhuat ,
Good question. Even though high concurrency is enabled in the workspace settings, each notebook still needs to explicitly join an available high concurrency session if you want them to share the same Spark app.
So yes, for your second notebook, you should select the available session like HC_Notebook_1_6782 manually — otherwise, it defaults to a standard session, which is why you're seeing it fail.
Also make sure:
- Both notebooks are in the same workspace.
- The high concurrency session is not at capacity (e.g. Joined: 1/5 means 4 more can join).
- You're not mixing different Spark pool types or configurations between notebooks.
Let me know if you want help setting up a shared session programmatically.
If my response resolved your query, kindly mark it as the Accepted Solution to assist others. Additionally, I would be grateful for a 'Kudos' if you found my response helpful.