Automation of Monitoring Scheduled Pipelines and Notebooks

[liney_marcela]

Hi Community,

I need help creating a notebook that can perform the following activity: Currently, the pipelines and some notebooks are executed because we have manually scheduled them to run daily, weekly, etc., at certain times, with a start date and an end date. When the end date arrives (we have set several years in advance), the pipeline execution will stop.

At the moment, it is not possible to set an infinite end date so that the execution never stops.

I would like to create a pipeline that can search for all pipelines and notebooks across all workspaces, and for those with an active schedule, check when the end date is and send alerts when there are 1 month, 2 months, 3 months left until their end date, so that the execution period can be extended if necessary.

I have a Client ID and a Secret ID for authentication. Could you help me with this?

[lbendlin]

You can use the sempy library to harvest and evaluate that information, including the schedules.  sempy.fabric package | Microsoft Learn

You will also want to consider using the tenant inventory reports and/or Purview to report on that kind of meta data.