Forum Discussion
Resolver IIISelf Service Platform inside Fabric
Hi Fabric Community,
we are currently conceptualizing a template for a self-service platform inside Fabric for a client. The basic idea is to create a platform, where developers and analysts can access their data and data products in raw or refined shape, enrich or combine it, build new reports or queries on top of them and explore new possible ways of gaining insights on their data. It should basically serve as a "sandbox" which provides all data but doesn't allow them to meddle with the established ETL processes, essentially just branching off of it.
Their basic architecture is as follows:
- Medallion architecture within a single workspace
- Data products are provided in dedicated lakehouses ("platinum layer") which shortcut relevant tables from gold layer in a separate workspace
- Analysts can only access the platinum layer and cannot see or change the ETL processes in the medallion architecture from bronze through gold
We are facing the following challenges:
1. Which layers of the data is reasonable to provide in the self-service space?
2. How should the data be made available? shortcuts/copy processes/One Lake Access? Especially concerning security/data integrity.
3. Should version control be provided for the self-service space?
4. How can we make use of Domains/Domain controllers to administer this process?
We would appreciate any insights or shared experience on this topic. Thank you! 🙂
Hi ObungiNiels,
Thank you for your detailed query and for sharing your architecture. It's great to see such an interesting discussion taking place!
The answer provided by nilendraFabric was indeed correct. Additionally, I am including some more points to make it easier for you.
- Which Layers of Data to Provide in the Self-Service Space?
- Platinum Layer (Gold Shortcuts): As mentioned, providing access to the platinum layer ensures analysts work with clean, reliable data. This approach minimizes the risk of disrupting ETL processes.
- Silver Layer (Optional): If you decide to provide access to the silver layer, ensure you have robust governance policies in place to prevent misuse or misinterpretation of semi-processed data.
- Bronze Layer: Avoid exposing raw data to maintain data quality and consistency.
- How Should Data Be Made Available?
- One Lake Shortcuts: This method is efficient and cost-effective, ensuring centralized governance and performance.
- Direct Lake Mode: Utilizing Direct Lake mode for Power BI can provide real-time analytics, ensuring up-to-date insights for your analysts.
- Implement role-based access controls (RBAC) to ensure users only access authorized datasets.
- Use Microsoft Purview for data cataloging, lineage tracking, and sensitivity labelling to maintain compliance and transparency.
- Should Version Control Be Provided for the Self-Service Space?
- Yes: Version control is essential for accountability and traceability. Using Git-based workflows within Fabric notebooks or pipelines allows analysts to experiment safely while preserving rollback options.
- Encourage analysts to create composite models when augmenting existing semantic models with departmental data. This balances self-service flexibility with centralized governance.
- Maintain a clear distinction between production datasets (certified) and experimental ones (endorsed) using Fabric’s endorsement features.
- How Can Domains/Domain Controllers Be Used for Administration?
- Domain Ownership: Align datasets with business domains (e.g; finance, marketing) following a Data Mesh-inspired approach. Each domain team should manage its own data products while adhering to global governance policies.
I hope these additional points help you in conceptualizing your self-service platform. If this helps, then please Accept it as a solution and dropping a "Kudos" so other members can find it more easily.
Thank you.
4 Replies
- nilendraFabric
Super User
Hello ObungiNiels
Very intresting discussion.
Here are my two cents :
. Which Layers of Data to Provide in the Self-Service Space?
Platinum Layer (Gold Shortcuts): Provide access to the platinum layer, which consists of curated and business-ready data products derived from the gold layer. This ensures analysts work with clean, reliable data without affecting ETL pipelines or raw data integrity.
Silver Layer (Optional): For advanced users, consider limited access to the silver layer for more exploratory analysis. However, this requires stricter governance to prevent misuse or misinterpretation of semi-processed data.
Bronze Layer: Avoid exposing raw data (bronze layer) in self-service environments, as it often lacks business context and can lead to inconsistent insights or data quality issues.
2. How Should Data Be Made Available?
OneLake Shortcuts: Use OneLake shortcuts to provide seamless access to curated datasets in the platinum layer. This method avoids duplication, reduces storage costs, and ensures centralized governance while maintaining performance.
Direct Lake Mode: Leverage Fabric’s Direct Lake mode for Power BI to enable real-time analytics directly on Delta Lake files stored in OneLake. This ensures consistent and up-to-date insights for analysts.
Security Considerations:
Implement role-based access controls (RBAC) to ensure users can only access authorized datasets .
Use Microsoft Purview for data cataloging, lineage tracking, and sensitivity labeling to maintain compliance and transparency across the self-service space.
3. Should Version Control Be Provided for the Self-Service Space?Version control is essential for maintaining accountability and traceability in self-service workflows.
Use Git-based workflows within Fabric notebooks or pipelines to allow analysts to experiment safely while preserving rollback options.
Encourage analysts to create composite models when augmenting existing semantic models with departmental data. This balances self-service flexibility with centralized governance.
Maintain a clear distinction between production datasets (certified) and experimental ones (endorsed) using Fabric’s endorsement features4. How Can Domains/Domain Controllers Be Used for Administration?
Domain Ownership: Align datasets with business domains (e.g., finance, marketing) following a Data Mesh-inspired approach. Each domain team should manage its own data products while adhering to global governance policiesPlease see if this helps
Thanks 
v-ssriganeshCommunity Support
Hi ObungiNiels,
Thank you for your detailed query and for sharing your architecture. It's great to see such an interesting discussion taking place!
The answer provided by nilendraFabric was indeed correct. Additionally, I am including some more points to make it easier for you.
- Which Layers of Data to Provide in the Self-Service Space?
- Platinum Layer (Gold Shortcuts): As mentioned, providing access to the platinum layer ensures analysts work with clean, reliable data. This approach minimizes the risk of disrupting ETL processes.
- Silver Layer (Optional): If you decide to provide access to the silver layer, ensure you have robust governance policies in place to prevent misuse or misinterpretation of semi-processed data.
- Bronze Layer: Avoid exposing raw data to maintain data quality and consistency.
- How Should Data Be Made Available?
- One Lake Shortcuts: This method is efficient and cost-effective, ensuring centralized governance and performance.
- Direct Lake Mode: Utilizing Direct Lake mode for Power BI can provide real-time analytics, ensuring up-to-date insights for your analysts.
- Implement role-based access controls (RBAC) to ensure users only access authorized datasets.
- Use Microsoft Purview for data cataloging, lineage tracking, and sensitivity labelling to maintain compliance and transparency.
- Should Version Control Be Provided for the Self-Service Space?
- Yes: Version control is essential for accountability and traceability. Using Git-based workflows within Fabric notebooks or pipelines allows analysts to experiment safely while preserving rollback options.
- Encourage analysts to create composite models when augmenting existing semantic models with departmental data. This balances self-service flexibility with centralized governance.
- Maintain a clear distinction between production datasets (certified) and experimental ones (endorsed) using Fabric’s endorsement features.
- How Can Domains/Domain Controllers Be Used for Administration?
- Domain Ownership: Align datasets with business domains (e.g; finance, marketing) following a Data Mesh-inspired approach. Each domain team should manage its own data products while adhering to global governance policies.
I hope these additional points help you in conceptualizing your self-service platform. If this helps, then please Accept it as a solution and dropping a "Kudos" so other members can find it more easily.
Thank you.- v-ssriganesh
Hi ObungiNiels,
May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.
Thank you.
powerbihelp87Helper V
I am looking to set up best practice as well you mentioned this below.
OneLake Shortcuts: Use OneLake shortcuts to provide seamless access to curated datasets in the platinum layer. This method avoids duplication, reduces storage costs, and ensures centralized governance while maintaining performance.
Does this mean the Data team will create shortcuts into departmental workspaces. How does this work- are you proposing best practice is PBI Dataset or allow user to create shortcuts?
Issue for me is users are used to importing data via SQL so the new architecture is a big change, however the user base was never great at importing they would keep duplicating the same data over and over again
