Forum Discussion
Srisakthi
Super User
Super UserHigh Concurrency session in Data pipeline
Hello Everyone, Consider a scenario where I have 10 notebooks and a datapipeline which uses session tag to call all those notebooks, in this case how many high concurrency session will be created...
Took 7 notebooks here , attached to same lakehouse ,env and present in same workspace.
In my case it should ideally create2 sessions 1 session (for 5 notebooks) another for ( 2 notebooks) according to the document.
https://learn.microsoft.com/en-us/fabric/data-engineering/configure-high-concurrency-session-notebooks-in-pipelines
But in my case it has created 4 sessions (1 for 4 notebooks , 1 session for remaining 3 notebooks). which is strange. But grouping and reducing time in execution is working as expected.
SrisakthiSuper User
Super UserThank you ObungiNiels Anonymous for your valuable insights.
If we have more than 10 notebooks then it is aligining 5 notebooks per HC.
Regards,
Srisakthi