Improve Log Analytics in Fabric for pipelines, notebooks and other relevant Fabric assets

[ydaponte]

There is a way to replicate the way we can Monitor Apache Spark applications with Azure Log Analytics with Synapse in Fabric, but it works inconsistently - it is probably not supported yet. Notebooks are called often from pipelines and for Fabric pipelines there is no integration with Log Analytics at the moment.

The ask is to integrate with Azure Monitor/Log analytics - or a better solution could be to centralize such logs within Fabric (inside the Monitor Hub for example - something like a Fabric Log Analytics) and allow to perform the same ADX queries that are allowed in Log Analytics for example.

[oaladekomo]

Workspace monitoring in Fabric provides developers and administrators with detailed logs and performance metrics within their Fabric experience, to support efficient troubleshooting, query optimization, and minimizing data downtime. Learn more here:  https://learn.microsoft.com/en-us/fabric/get-started/workspace-monitoring-overview

Workspace monitoring currently supports multiple workloads, including Power BI (SemanticModelLogs), Eventhouse (query, command, and ingestion logs), and GraphQL APIs. Logs and metrics are surfaced via a read-only Monitoring Eventhouse and Eventstream, with updates arriving within 10–15 minutes of activity.

Support for all Fabric workloads will be added incrementally including for data factory, spark, airflows, etc.