topic Re: Snapshot of Notebook from Pipeline in real time in Data Engineering

Snapshot of Notebook from Pipeline in real time

spark123 — Wed, 11 Dec 2024 14:49:27 GMT

Hello,

Why can't we see the snapshot of the notebook's execution In real time.

Re: Snapshot of Notebook from Pipeline in real time

Anonymous — Thu, 12 Dec 2024 02:32:07 GMT

I ran a test and while I can not view snapshots of notebook activity directly in pipeline.

I have a workaround, you can use Monitor Hub to track all activity in your workspace. In this you can view the Notebook Execution Snapshot.

If you have any other questions please feel free to contact me.

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

Re: Snapshot of Notebook from Pipeline in real time

spark123 — Thu, 12 Dec 2024 14:29:26 GMT

Hello Yang @Anonymous ,

Thank you for your response. However, it doesn't fully address my concern. While it’s true that we can view the snapshot after the execution of the notebook activity, my question pertains to observing the snapshot during the execution itself to monitor what’s happening in real time.

Looking forward to your guidance on this.

Best regards,

Re: Snapshot of Notebook from Pipeline in real time

Anonymous — Fri, 13 Dec 2024 07:33:53 GMT

Hi @spark123 ,

Currently, Fabric does not support viewing snapshots of notebook execution in real time in the pipeline. This does make real-time monitoring a bit difficult.

One workaround is to monitor real-time execution through status updates of Notebook activity in Monitor Hub.

This way, even if you can't directly view the snapshots in real-time, you can still get an idea of the execution through the different states.

If you have any other questions please feel free to contact me.

Best Regards,
Yang
Community Support Team