Deployment Pipelines: preserve/override schedules per environment (no overwrite on deploy)

[JP8991]

When deploying from Dev/Non-Prod to Prod, the target’s pipeline schedule is overwritten. Please add deployment rules (or an opt-out) so schedules can be environment-specific without showing as drift in comparisons. This avoids Prod cadence being reset and reduces capacity burn in Non-Prod.

[grisensko]

This should be a BUG, they changed this without telling anyone and cause problem to everyone deploying pipeline. Like ADF, the Triggers Deployment should be a separate entity

[zhaya]

Hi @JP8991, thanks for the feedback. 

I’d like to confirm the usage scenario with you. From what I understand, you’ve deployed the same pipeline in both Dev and Prod environments, but you want each to follow its own distinct schedule. You’re looking for a way during CICD to ensure that each environment’s pipeline uses its predefined schedule without being overridden by the CICD process. Is that correct?

Also, could you confirm whether the “Schedule” you’re referring to is the Fabric scheduler accessed via the side panel—the one that allows multiple schedules to be configured?

[grisensko]

@zhaya We are not using CICD but using basic Fabric Deployment Pipeline. We move Pipeline from Dev/Test/Prod this way. In prod we have specific production load schedule. Ex: Every morning, every 4 Hours, some pipeline are even run every 15 min. Etc .. In dev/Test we cannot stress the system that much and read from source at the same frequency.. so we have less aggressive schedule. It use to work fine.. I could have a once a week schedule in test, and my prod schedule daily. Schedule was not deployed using the deployment pipeline. Now we lost multiple prod schedule because the test one.. and sometimes dev one are pushed automatically in production.

[JP8991]

@zhaya my issue is as @grisensko describes. We are using the schedule as you say from the side panel menu.

[zhaya]

@JP8991  @grisensko  thanks for the explaination! I’ve documented the usage scenario you described and will explore optimizations tailored to it. The detailed solution will be shared once it’s finalized. Thank you so much for your valuable feedback!

[KeithRimington]

This is super frustrating. I don't have the CUs to spend to run all of my loads in every dev/test/prod environment at the production cadence. Nor can the source systems (which do not always have a dev/test/prod) take the burden of getting hit by three extracts simultaneously. Schedules should either be excluded from the deployment pipelines entirely, or at least be given a deployment rule to ignore schedules. I don't have a single case across all of my integrations where I want the dev/test/prod schedules to be identical.

[samloud]

Totally agree with OP. Deploying the schedule as part of the pipeline itself is ludicrous. "I don't have a single case across all of my integrations where I want the dev/test/prod schedules to be identical." *Exactly*

[zhaya]

@KeithRimington @samloud Thanks for the comment.

We’re planning to use the Variable Library to control the scheduler’s On/Off state across different workspaces/envs. This would allow users to disable the scheduler by assigning different values, during the deployment. Do you think this approach would adequately address the issues you’ve encountered?

I’m also curious—you mentioned that job execution requirements vary across environments. What if we enable users to predefine Scheduler configurations via VarLib? For example, in Prod, it could run daily, while in Test, it runs every Monday and Wednesday. Users could then apply different preset templates to schedulers in different environments.

Here’s how the full flow might look:

1. A user configures the scheduler in the Test environment and deploys it to Prod via the CICD/deployment pipeline.
2. The schedulers in Test and Prod each use different Variable Library templates for configuration.

How do you feel about this approach?

[grisensko]

@zhaya Personnaly, I find that having to rely on Env. Variable to configure the Schedule is too complex. It's better than nothing of course. Often in our use case Schedule was only in Production and only known / determined by Prod Responsible/Administrator. Having to go trough deployment of the Env. Library involve all schedule adjustement need to be done in Dev. and Deployed in order to simply Test/re-enable Test schedule. Why don't you simply remove the Schedule from deployement? like other said : "I don't have a single case across all of my integrations where I want the dev/test/prod schedules to be identical." Same For us!! We have close to 200 Workspaces! 15 Data Domains, a ton of bussiness report workspace. A ton of different schedule/use case and in absolutely none of our use case having the schedule part of CI/CD give us anything.

[JP8991]

Just echoing and reiterating the points above. The schedule should be removed from the deployment pipeline metadata. There is absolutely no reason why you would want your test, dev and prod workloads on the same schedule. The compute, and like others have mentioned, the hit on source systems is just not necessary, and in some cases not possible.