Data Pipelines for cross-workspace medallion execution

Hi seanrich ,

Yes — use the "master orchestrator" pattern.

Create a fourth workspace called Orchestration or DataOps. This is your control plane.

Each medallion workspace (Bronze, Silver, Gold) keeps its own Lakehouses, Notebooks, Variable Libraries, and one or more unscheduled worker pipelines (e.g., Bronze_Ingest_Main). These never run on their own — they only fire when called.

The Orchestration workspace holds your master pipeline, which uses the Invoke Pipeline activity to call each worker sequentially: Bronze → Silver → Gold. It also holds your schedule triggers, failure alerting logic, and optionally a small Lakehouse for audit logging.

Why not put the master in one of the medallion workspaces? Three reasons: capacity isolation (a heavy Bronze ingestion won't starve your orchestrator), security segmentation (not everyone who edits Bronze notebooks should control the master schedule), and clean separation of concerns (Bronze shouldn't implicitly "own" the whole chain).

Since you're on a clean-slate tenant, also plan your environment strategy early — decide whether you'll have Dev_Bronze, Dev_Silver, etc. or use deployment pipelines to promote, because that multiplies your workspace count fast.

This pattern scales well and keeps the mental model obvious for anyone joining the team.

If this post helps, then please appreciate giving a Kudos or accepting as a Solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let me know. Thanks a lot!

Hi seanrich ,

One orchestration workspace per source system (orch-salesforce, orch-replicon, etc.). Each one owns the sequential bronze → silver → gold chain for its source. They run independently, potentially on different schedules — maybe Salesforce is near-real-time while Replicon is a nightly batch. This keeps the blast radius small: a Replicon failure doesn't block Salesforce processing, and whoever owns a source system manages its own cadence.

Then one Platform Orchestration workspace sits above everything. This is your thin coordination layer — it doesn't micromanage notebook execution. Its responsibilities are:
• Trigger domain-layer refreshes (ticketing-shortcuts, sales-opps-shortcuts, etc.) once the relevant upstream sources have landed
• Trigger audience workspace refreshes (finance, sales, customer-support) after their dependent domains are complete
• Centralized alerting, audit logging, and health dashboards

Why not one giant orchestrator? At this scale you'd have dozens of Invoke Pipeline activities, different sources needing different schedules, and one failure potentially blocking unrelated downstream work. Why not an orchestrator per workspace? Too granular — you'd lose the ability to express "after all sales-related sources land, refresh the sales-opps domain."

On the environment strategy — since you mentioned UAT and prod via additional workspaces, I'd strongly recommend using Fabric deployment pipelines to promote content rather than manually maintaining separate dev/uat/prod copies of every workspace. That multiplies your workspace count fast and creates drift risk.

So your prod workspace inventory would look roughly like:
• Source orchestration: orch-salesforce, orch-replicon, etc.
• Source medallion: bronze/silver/gold per source (as you have)
• Domain: as you've outlined
• Audience: as you've outlined
• Platform orchestration: one workspace at the top

Clean, independently securable, capacity-isolated, and easy for someone new to reason about.

If this post helps, then please appreciate giving a Kudos or accepting as a Solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let me know. Thanks a lot!