Forum Discussion

etane's avatar
etane
Icon for Helper V rankHelper V
7 months ago
Solved

Stop Refresh if it Exceeds X Minutes

Hello.   I have lots of dataflows that are linked.  And, they are scheduled to refresh 30 minutes apart.  It usually takes about 5 minutes for a refresh to complete.  However, sometimes, for unknow...
  • Natarajan_M's avatar
    7 months ago

    Hi etane , 

    There is no built-in method to check/stop the refresh time of any object in fabric . 

    you can use either pipeline/notebook to orchestrate the entire process of refreshing and monitoring the pipelines .

    Note bookapproach :

    Run A → wait max 30 minutes  max
                  If completed before 30 mins → Run B
                  If exceeded 30 mins → cancel + notify + STOP A -> go ahead with B

    Run B → wait max 30 minutes  max
                  If completed before 30 mins → Run C
                  If exceeded 30 mins → cancel + notify + STOP B -> go ahead with C
    Run C → wait max 30 minutes  max
                  If completed before 30 mins → notify user all process compelted
                  If exceeded 30 mins → cancel + notify + STOP C 

    Use the Rest API calls to get the dataflow run status

    Also any reason for running the data flows in sequential? is there any dependency or just to limit the stress on the capacity the dataflows are ran in sequential

    Thanks