Single Version of Truth Mapping for Multiple Data Using a Single Notebook

[Mirdula]

Hello everyone,

My team is in the process of implementing a Medallion architecture in Microsoft Fabric, ingesting data from multiple ERP systems. One of these systems will serve as the “master,” and all dimensions must align with a Single Version of Truth (SVOT) based on its schema.

So far, we have ingested and transformed data from one source system, which includes both Finance and Sales data. Although the core SVOT mapping logic is nearly identical across functions, there are a handful of dimensions that require function-specific handling.

Today, we maintain separate PySpark notebooks for each dataset (Finance, Sales, etc.) to apply the SVOT mapping. We’d like to consolidate this into a single, parameterized PySpark notebook that can handle all functional datasets while still accommodating their slight dimension differences. 
 

Does anyone have recommendations on how we could design and orchestrate one reusable SVOT mapping notebook in Microsoft Fabric?

[nilendraFabric · Accepted solution]

Hello @Mirdula 

not official MS recommendations, but these are few which I try to follow in my deployments 

Centralize SVOT Logic: Keep your mapping logic modular and in shared modules to avoid duplication.
• Parameterize Notebooks: Use parameters for dataset type, source/target paths, and configuration settings.
• Handle Function-Specific Needs: Apply conditional logic or configuration-driven mappings for each functional dataset (e.g., Finance, Sales).
• Orchestrate with Pipelines: Use Microsoft Fabric pipelines or notebook widgets to pass parameters and manage execution.
• Follow Medallion Architecture: Structure your data into Bronze (raw), Silver (SVOT-mapped), and Gold (curated) layers.
• Prioritize Maintainability: Use the DRY (Don’t Repeat Yourself) principle so updates are easy and consistent across datasets.

[Mirdula]

Thank you for your response @nilendraFabric 🙂 . It helps 👍