Forum Discussion

JibinSebastian's avatar
JibinSebastian
Icon for Advocate II rankAdvocate II
4 months ago
Solved

Scheduling a Microsoft Fabric Pipeline with Specific Time Intervals

I would like to configure a Microsoft Fabric pipeline to run on a recurring schedule with the following requirements:

  • Execute every 10 minutes
  • Start time: 7:10 AM EST
  • End time: 6:50 PM EST
  • Frequency: Monday through Friday only

Could you please provide guidance on how to implement this schedule within Microsoft Fabric, including the appropriate configuration (e.g., cron expression or trigger settings) to meet these requirements?

  • deborshi_nag's avatar
    deborshi_nag
    4 months ago

    Hello JibinSebastian 

     

    I am assuming your Fabric capacity is paused outside business hours (7AM - 7PM) to save costs. If you set your pipeline to trigger every 10 mins, the trigger will run fine but your pipeline will fail because your capacity is found Paused. 

     

    Secondly, Power BI semantic model enforces a 30 min minimum interval window, so the service backend will block it, regardless we tried a 10 min interval. Maximum 8 scheduled refreshes per day per dataset is allowed. 

     

    Considering the above, I suggest you design it the following way:

     

    Orchestrator pipeline

    Notebook activity (checks against weekends
    Scheduled every 10 minutes, business hours only

    Pipeline 1 – Data ingestion (trigger if parent Notebook passes as True)
    Copy Data

    Pipeline 2 – Semantic model refresh (runs after pipeline 1 success, engine may skip or throttle if run within 30 mins)
    Semantic model refresh activity

     

    Schedule as discussed in previous post:

    • Frequency: every 10 minutes
    • Time window: 07:10–18:50
    • Time zone: EST

     

10 Replies

  • Hello JibinSebastian 

      

    Unfortunately Fabric pipelines currently don't have support for cron expression or weekday only trigger. What you could use is set one trigger that covers rest and apply the weekday logic inside a Notebook in the pipeline. 

     

    So, effectively your pipeline trigger would look something like this:

    Frequency: Minute - Interval: 10 - Time zone: Eastern Standard Time - Start time: 07:10 - End time: 18:50
     
    Your notebook logic could include something like this:
     
    # skip if it is not a weekday

    if now.weekday() >= 5:
        print("Pipeline not scheduled to run on weekends – exiting ...")
        dbutils.notebook.exit("SKIPPED")
     
    • JibinSebastian's avatar
      JibinSebastian
      Icon for Advocate II rankAdvocate II

      Hi @debroshi, thank you for your response.

      I currently have two pipelines:

      1. The first pipeline uses Copy Data to move a small amount of data from a large SQL Server table.
      2. The second pipeline is for refreshing a semantic model.

      I’m new to Microsoft Fabric and ran into a couple of issues:

      • I tried scheduling the pipeline to run every 10 minutes. It works during active hours, but when Fabric is suspended (outside 7 AM to 7 PM), the pipeline fails to execute.
      • For the semantic model refresh, the scheduler only allows a minimum interval of 30 minutes. However, I need it to refresh every 10 minutes on weekdays between 7 AM and 7 PM.

      Could you please guide me on how to set up the trigger properly? It would really help if you could explain it in simple, beginner-friendly terms.

      Thank you!

      • deborshi_nag's avatar
        deborshi_nag
        Icon for Super User rankSuper User

        Hello JibinSebastian 

         

        I am assuming your Fabric capacity is paused outside business hours (7AM - 7PM) to save costs. If you set your pipeline to trigger every 10 mins, the trigger will run fine but your pipeline will fail because your capacity is found Paused. 

         

        Secondly, Power BI semantic model enforces a 30 min minimum interval window, so the service backend will block it, regardless we tried a 10 min interval. Maximum 8 scheduled refreshes per day per dataset is allowed. 

         

        Considering the above, I suggest you design it the following way:

         

        Orchestrator pipeline

        Notebook activity (checks against weekends
        Scheduled every 10 minutes, business hours only

        Pipeline 1 – Data ingestion (trigger if parent Notebook passes as True)
        Copy Data

        Pipeline 2 – Semantic model refresh (runs after pipeline 1 success, engine may skip or throttle if run within 30 mins)
        Semantic model refresh activity

         

        Schedule as discussed in previous post:

        • Frequency: every 10 minutes
        • Time window: 07:10–18:50
        • Time zone: EST

         

  • Kagiyama_yutaka's avatar
    Kagiyama_yutaka
    Icon for Continued Contributor rankContinued Contributor

    I think u just run the 10‑min window and let the first Notebook line quietly step out on weekends, ’cause Fabric’s trigger can’t keep weekday‑state at dispatch and this path stays the most stable in real runs.

    • JibinSebastian's avatar
      JibinSebastian
      Icon for Advocate II rankAdvocate II

      Could you please explain a bit on this.

      I didn't quite understand that

      • Kagiyama_yutaka's avatar
        Kagiyama_yutaka
        Icon for Continued Contributor rankContinued Contributor

        sure. I just meant Fabric’s trigger can only fire every 10 min and it can’t check weekdays or time windows by itself. so let the trigger run, and add one line in the Notebook to skip if it’s weekend or outside 7:10–18:50.

  • Hi JibinSebastian

    Thank you for reaching out to the Microsoft Fabric Community Forum. Also, thanks to Kagiyama_yutakalbendlinlbendlindeborshi_nag,  for those inputs on this thread.

    Has your issue been resolved? If the response provided by the community member Kagiyama_yutakalbendlinlbendlindeborshi_nag, addressed your query, could you please confirm? It helps us ensure that the solutions provided are effective and beneficial for everyone.

    Hope this helps clarify things and let me know what you find after giving these steps a try happy to help you investigate this further.

    Thank you for using the Microsoft Community Forum.

  • Hi JibinSebastian ,

    We haven’t received any update from your side. Please let us know if you need any additional details or clarification.

     

    Thank you.