Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
1 year ago
Solved

Fabric Security Issues

We’re trying to implement a medallion architecture in Microsoft Fabric and are running into major roadblocks with access control and workspace security inheritance.

 

Architecture Goals

  • We want to set up bronze layer ingestion pipelines in an Ingestion Workspace.
  • Shortcut the bronze tables into a separate Curated Workspace, where a different team:
    • Transforms data into silver/gold layers.
    • Builds views and semantic models.
    • Manages data access (eventually with RLS/CLS).
  • Ensure the curated team cannot modify raw/bronze layer data but can read from it and build on top.

 

Current Issues

  • Shortcuts Require Access to Source Lakehouse
    • Only lakehouses support shortcuts (not warehouses).
    • To use shortcuts in the curated workspace, users must be granted access to the source lakehouse in the ingestion workspace.
    • This defeats the purpose of separating roles and delegating security management to the curated team, because we can’t isolate source data access.
  • Workspace Role Overrides Lakehouse Permissions
    • Even if we assign "Read" role on the lakehouse, if users are Members of the workspace, they can insert, update, delete, or alter lakehouse tables.
    • We want the curated team to be Members of their workspace (so they can build models, views, and manage sharing), but not able to change source data.
  • No Way to Enforce Read-Only in Lakehouse
    • There is currently no mechanism to enforce read-only access to a lakehouse for workspace members.
    • Table-level properties cannot restrict DML operations.
    • So we cannot safely use the lakehouse for raw data without risk of accidental data corruption

 

Current Workaround (Not Ideal)

  • We maintain ingestion pipelines in the ingestion workspace.
  • We ingest data directly into a lakehouse in the curated workspace.
  • We’ve added RLS to the SQL endpoint, and are instructing curated data developers to only use the SQL endpoint, where they can’t execute DML by default.
  • However, this is not scalable or robust. There's still no way to enforce this policy at the platform level.

 

What We Need

  • A way to truly enforce read-only access to lakehouses, regardless of workspace role.
  • Ideally:
    • Shortcuts would allow us to delegate security only at the target lakehouse.
    • Or, Microsoft introduces lakehouse-level DML restrictions that override workspace membership.

 

Questions

  • Is there a way to prevent data manipulation language operations in a lakehouse (INSERT, UPDATE, DELETE, ALTER) to members of a workspace (override priviledges granted to the workspace member role)?
  • Is there now a way — or is there going to be a way — to delegate security of shortcutted tables to the target lakehouse, thereby removing the need to grant access to the source lakehouse?
  • Does Microsoft Purview allow us the ability to override workspace membership and grant users read access and the ability to create views in a lakehouse without enabling full modification rights? (Note: We don’t manage M365, only Fabric, and currently don’t have access to Purview. Would this be an option for us if our M365 team enables it? Don't want to go down that path until I know it will do what we need it to.)

 

Thanks in advance for any insight.

  • Hi Anonymous ,

    Thanks for reaching out to the Microsoft fabric community forum.
    suparnababu8  Thanks for your prompt response

    Anonymous ,

    Potential Workarounds
    Given the current limitations, here are some strategies you might consider:

    Separate Workspaces for Different Layers:

    Maintain separate workspaces for ingestion (bronze) and curated (silver/gold) layers.

    Restrict access to the ingestion workspace to only service principals or specific users responsible for data ingestion.

    Grant the curated team access only to the curated workspace, ensuring they cannot modify raw data .

    Use of Viewer Role with SQL Endpoint:

    Assign the Viewer role to users in the curated workspace.

    Instruct users to access data exclusively through the SQL analytics endpoint, which enforces read-only access.

    Note that this approach limits users from using notebooks or the Lakehouse UI for data exploration.

    Implement Row-Level and Column-Level Security:

    Within the curated workspace, apply Row-Level Security (RLS) and Column-Level Security (CLS) to control data access at a more granular level.

    This ensures that even if users have broader access, they can only view data they're authorized to see.

    Monitor and Audit Activities:

    Implement monitoring and auditing to track data access and modifications.

    While this doesn't prevent unauthorized changes, it can help in identifying and addressing them promptly.

    Lakehouse sharing and permission management - Microsoft Fabric | Microsoft Learn
    Give users access to workspaces - Microsoft Fabric | Microsoft Learn
    Implement medallion lakehouse architecture in Fabric - Microsoft Fabric | Microsoft Learn

    If this post helped resolve your issue, please consider giving it Kudos and marking it as the Accepted Solution. This not only acknowledges the support provided but also helps other community members find relevant solutions more easily.

    We appreciate your engagement and thank you for being an active part of the community.

    Best regards,
    LakshmiNarayana.

4 Replies

  • Hi Anonymous 

     

    Thank you for detailed explained of your problem. Let me answer your questions as follows

     

    • Is there a way to prevent data manipulation language operations in a lakehouse (INSERT, UPDATE, DELETE, ALTER) to members of a workspace (override priviledges granted to the workspace member role)?

     At present microsoft fabric, doen;t support for INSERT,UPDATE,DELETE and ALTER operations and also not provide read only access to the lakehouse of workspace roles/memebers. For example if you shared a  lakehouse with a member having read access, but if the user has Member role , then thatuser can be able to modify the tables(by using spark). If you want to restrict the user, you can inplemet RLS or CLS. read more detaisl here Microsoft Fabric end-to-end security scenario - Microsoft Fabric | Microsoft Learn

     

    • Is there now a way — or is there going to be a way — to delegate security of shortcutted tables to the target lakehouse, thereby removing the need to grant access to the source lakehouse?

    Currently, if fabric users want to use shortcuuts in Fabric, they must have permision to access the original lakehouse. This makes it to difficult to seperate roles properly. There is not a way to give security permiissions only for the target lakehouse without giviing access to the sorurce lakehouse. Microsoft introduced, to create secuirity settings specifiically for shortcuts, allowing users to access them without needing permision for the original lakehouse.

     

    • Does Microsoft Purview allow us the ability to override workspace membership and grant users read access and the ability to create views in a lakehouse without enabling full modification rights?

      Yes, Purview helps with Govenrnace and security, but it's not clear where you can override workspace  memership roles in Fabric. If your organisation uses Purview, it will help you manage data access polices, but it will not completely fix the issue of inherited workspsce. Since you are not managing M365, you can check with your M365 Team  to see if Purview can enfore reda only access.

     

    So, Use SQL analytics point in lakehosue only for read only access, bcz you can able to performa DML operations. Use seperate workspaces for different roles. If you want to introduce DML operations in Lakehouse you can submit your idea here Fabric Ideas - Microsoft Fabric Community

     

    Thank you again!

     

    Did I answer your question? Mark my post as a solution!

    Proud to be a Super User!

     

     

    • Anonymous's avatar
      Anonymous
      Not applicable

      No, this did not answer my question.

       

      To your first point:

      • Even the link you provided contains inaccurate information/guidance (https://learn.microsoft.com/en-us/fabric/security/security-scenario). It says, "Your data analysts need access to the second lakehouse in the gold workspace. They'll connect to the SQL analytics endpoint of that lakehouse to write SQL queries and perform analysis. So, you share that lakehouse with them and provide access only to objects they need (such as tables, rows, and columns with masking rules) in the lakehouse SQL analytics endpoint by using the SQL security model. Data analysts can now only access data that's relevant to their role and department and they can't access the other items in the workspace, such as the pipelines and notebooks" which is inaccurate because you can't just give access to the "second" lakehouse (which contains shortcuts) without granting access to the original lakehouse. Again, this is an issue.
      • Secondly, we already understand how to set up RLS/CLS security but this doesn't allow us to restrict users from modifying tables in a lakehouse. It will only show them the data they have access to but if they are a member/contributor of the workspace, they can still go into the lakehouse and modify tables. So, RLS/CLS does not have relevance with regard to my original question.
      • As a side note: you would think that more granular (item-level permissions) would override workspace permissions but that does not appear to be the case.  

      To the second point:

      • I believe you are saying Microsoft introduced security setting specifically for shortcuts, can you provide more information on that (link to more information, etc.)?

      And finally, to your statement about using SQL analytics endpoint so I CAN perform DML operations…I want to RESTRICT DML operations (not find a way to perform them). And you are inaccurate in that statement as well because in stand-alone warehouses, you can perform DML operations but in the SQL End Point of a lakehouse (warehouse tied to a lakehouse), DML operations are restricted by default. However, DML operations can still be performed on the lakehouse if you have above viewer access and there is no way currently (from what I can tell) to restrict it which again is what I am trying to do.

  • v-lgarikapat's avatar
    v-lgarikapat
    Community Support

    Hi Anonymous ,

    Thanks for reaching out to the Microsoft fabric community forum.
    suparnababu8  Thanks for your prompt response

    Anonymous ,

    Potential Workarounds
    Given the current limitations, here are some strategies you might consider:

    Separate Workspaces for Different Layers:

    Maintain separate workspaces for ingestion (bronze) and curated (silver/gold) layers.

    Restrict access to the ingestion workspace to only service principals or specific users responsible for data ingestion.

    Grant the curated team access only to the curated workspace, ensuring they cannot modify raw data .

    Use of Viewer Role with SQL Endpoint:

    Assign the Viewer role to users in the curated workspace.

    Instruct users to access data exclusively through the SQL analytics endpoint, which enforces read-only access.

    Note that this approach limits users from using notebooks or the Lakehouse UI for data exploration.

    Implement Row-Level and Column-Level Security:

    Within the curated workspace, apply Row-Level Security (RLS) and Column-Level Security (CLS) to control data access at a more granular level.

    This ensures that even if users have broader access, they can only view data they're authorized to see.

    Monitor and Audit Activities:

    Implement monitoring and auditing to track data access and modifications.

    While this doesn't prevent unauthorized changes, it can help in identifying and addressing them promptly.

    Lakehouse sharing and permission management - Microsoft Fabric | Microsoft Learn
    Give users access to workspaces - Microsoft Fabric | Microsoft Learn
    Implement medallion lakehouse architecture in Fabric - Microsoft Fabric | Microsoft Learn

    If this post helped resolve your issue, please consider giving it Kudos and marking it as the Accepted Solution. This not only acknowledges the support provided but also helps other community members find relevant solutions more easily.

    We appreciate your engagement and thank you for being an active part of the community.

    Best regards,
    LakshmiNarayana.

    • v-lgarikapat's avatar
      v-lgarikapat
      Community Support

      Hi Anonymous ,

      If your issue has been resolved, please consider marking the most helpful reply as the accepted solution. This helps other community members who may encounter the same issue to find answers more efficiently.

      If you're still facing challenges, feel free to let us know we’ll be glad to assist you further.

      Looking forward to your response.

      Best regards,
      LakshmiNarayana.