Stream Notebook logs

[AdarshPanasri]

Hello Everyone, I have setup a logging mechanism using the logging library from python, while my code generates the logs and persist them in the lakehouse, I want to stream the logs real-time so that I can monitor the status of the job, has someone tried something similar and can suggest something that I can implement to achieve this. Thanks 

[vgiatti]

You can use the method proposed by Dennes. See:

• Solved: Notebook Logs - Microsoft Fabric Community
• https://youtu.be/F8w59GYDEBA?si=QYTsCtcQ3q_FxiYj

the real pain is that (as of today) the tables are ingested in Azure Log Analytics as "Custom Table (classic)", and these tables cannot be added to an Azure Events Hub, which is the only "make-sense" real-time source to be used in a Fabric Eventstream pipeline.

[Srisakthi]

Hi @AdarshPanasri ,

Not sure whether a direct way available as of now, but you can try this option

Go to your notebook, select Run-> Viewallruns and then go to your specific application , you can see as below

There should be some api's available to fetch the logs, but I havent explored. Thought this would give some information.

Regards,

Srisakthi

[Anonymous]

Hi @AdarshPanasri ,

Is my follow-up just to ask if the problem has been solved?

If so, can you accept the correct answer as a solution or share your solution to help other members find it faster?

Thank you very much for your cooperation!

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

[Anonymous · Accepted solution]

Hi @AdarshPanasri ,

I have not found a way to stream logs in real time in Fabric, so I'd be grateful if others could advise.

Those who want to monitor the status of jobs can use the Microsoft Fabric Monitoring Center, where users are able to monitor Microsoft Fabric activity from a central location.

By clicking on the Monitor pane you can select the type of filter you want and then you can see its running status and also open it to view the details.

More details about Monitor can be found in this document:

Use the Monitor hub - Microsoft Fabric | Microsoft Learn

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!