Inquiry about Microsoft Fabric Notebooks Performance and Configuration

[Adlane_Kadri]

I'm reaching out to inquire about the performance and configuration of Microsoft Fabric Notebooks. I've been experiencing significant delays across all my code executions, even for simple print commands. I would like to understand if this delay is associated with my account's current capacity or if there are specific configurations to enhance overall responsiveness.

Additionally, I'm seeking clarification on whether running the same notebook in multiple pipelines simultaneously could potentially cause issues. Specifically, I want to ensure that each pipeline can initiate its own instances of the notebook, allowing for independent parameter usage.

Thank you

[v-cboorla-msft]

Hi @Adlane_Kadri 

Thanks for uisng Microsoft Fabric Community.

Apologies for the issue that you are facing here.

Possible reasons for the issue are:

Exceeding your allocated resources (CPUs, memory) can lead to throttling and slower execution times. Check your account's capacity limits and usage metrics to see if you're approaching or exceeding them. Use the Fabric Capacity Metrics app to monitor your CPU, memory, and storage usage. If you're consistently exceeding your allocated resources, consider upgrading your account or optimizing your code to reduce resource consumption.

The other reason might be due to throttling. Throttling occurs when a tenant’s capacity consumes more capacity resources than it has purchased. Too much throttling can result in a degraded end-user experience. A Fabric tenant can create multiple capacities and assign workspaces to a specific capacity for billing and sizing.

For more info please refer Link1, Link2.

Yes, running the same notebook in multiple pipelines simultaneously in MS Fabric can potentially cause issues.

For further more details please refer this thread Link.

I hope this information helps. Please do let us know if you have any further questions.

Thanks

[Adlane_Kadri]

Hi @v-cboorla-msft ,

Thank you for your prompt response and the provided insights into potential causes for the performance issues.

I understand the importance of monitoring resource usage, and I will certainly explore the Fabric Capacity Metrics app to assess and optimize resource consumption on my end.

However, I'd like to express that it seems unusual for a simple notebook with basic commands, such as a print statement (e.g., print("Hello world")), to experience significant delays. Even when utilizing a better resources, the delays persist, raising concerns about the overall responsiveness.

As I explore potential solutions to address this issue, I am open to any additional guidance or recommendations you may have to enhance the notebook's performance.

Regarding running the same notebook in multiple pipelines, I appreciate your clarification on potential issues. To further elaborate on my use case, I aim to avoid duplicating notebooks for each pipeline when updating a lakehouse/table (just an example). Instead, I want to use a single notebook and dynamically pass the table name as input from each pipeline, ensuring independent instances are created for each execution.

If you could provide any additional guidance on optimizing notebook performance and achieving the described use case, it would be greatly appreciated.

Thank you again for your assistance.

Best regards,

[v-cboorla-msft]

Hi @Adlane_Kadri 

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
In case if you have any resolution please do share that same with the community as it can be helpful to others.
Otherwise, will respond back with the more details and we will try to help.

Thanks