Fabric Compliance & Risk Audit Solution
1 Summary
How often do you receive a notification that your medical data may have been exposed? Breaches happen for many reasons, but a common - and preventable - one is code that accidentally leaks sensitive information: hardcoded credentials, tokens, connection strings, or PHI/PII printed to logs and outputs. In fast-paced delivery environments, these risks are easy to overlook and expensive to rectify later.
Fabric Compliance & Risk Audit addresses this gap by adding an automated compliance audit layer focused on the code surface area in Microsoft Fabric. It continuously scans notebooks for high-risk patterns and enforces configurable policies aligned to standards such as HIPAA. The solution complements Fabric’s existing security stack - Purview, Key Vault, ACL/RBAC, and sensitivity labels - by ensuring that the code itself follows secure-by-design practices and remains audit-ready.
Demo prototype link: https://sec-audit-fabric.figma.site/
Video Demo: https://www.youtube.com/watch?v=VVpQDq1sd4o
2 Goals and Success Metrics
Goal #1: Minimize sensitive data exposure risk
- 0 critical findings in production (hardcoded secrets, secrets in logs, secrets written to storage)
- ≥ 95% coverage of in-scope notebooks; MTTR (Mean Time to Recovery) for critical findings ≤ 72 hours
Goal #2: Provide scheduled "antivirus-like" scanning for Fabric code
- Scheduled + on-demand scans
- Actionable findings (severity, exact location, fix guidance) with low false positives
- Built-in HIPAA policy packs + configurable custom rules
Goal #3: Be cost-effective and scalable
- Predictable cost per workspace / per scan
- Incremental scanning to minimize compute and API usage
3 Problem Statement
Early-stage projects often embed passwords, API keys, tokens, and connection strings directly in notebooks or logs. As notebooks get copied and archived into shared folders, these exposures spread and become harder to control.
Best practices and code reviews help, but don’t scale consistently. We need an automated, continuous way to detect and prevent these risks across Fabric workspaces. This solution provides that via an easy-to-use security scanner powered by LLMs and configurable rules, adapted to the most popular standards, such as HIPAA.
4 Who It's For
This solution is for the companies that care about (1) their future and (2) their customers' privacy. Especially, it may be critical for companies that work with sensitive information, such as those in the healthcare sector. Particular roles who may be interested: Tech Lead, IT Director, Security Compliance Officer.
5 Key Features & Use Cases
- The tech lead configures a night run of the audit to detect potential vulnerabilities, to discuss it with the team during a daily meeting the next day to solve and prevent data leaks.
- The IT Director and/or Security Compliance Officer runs it with a necessary frequency to detect more global vulnerabilities to build a secure and compliance-ready environment.
Main workflow:
- Create a desired state of the system in terms of security & privacy.
- Choose the core Fabric Workspaces to start with.
- Configure the solution, installing it from the Gallery and configuring the LLM.
- Signing the HIPAA Business Associate Agreement with Microsoft to get the secure and HIPAA-compliant LLMs
- Configuring the Fabric Compliance & Risk Audit Solution to satisfy the desired future state of the Fabric environment
- Monitor the results, investigate the found issue, and prevent them in the future
6 Solution Overview
Prompt sample, to demonstrate LLM is very capable of solving this type of task:
You are a security auditor. Your goal is to return the JSON with the result on the following policies in the Python notebook.
Policy ID: 1
Policy name: API Keys
Rule: No API keys are in the code as is
Policy ID: 2
Policy name: Credentials
Rule: No passwords or connection strings in the code.
Your goal is to check the code below and return a JSON in the format:
[
{
“policy_id”: <policy_id>,
“evaluation”: <red - if critical issue, yellow - if non-critical issue, green - if no issue>,
“details”: <details why if the subject is found, otherwise empty string>
},
…
]
Code:
<<<
import requests
import logging
# Hardcoded secrets (API key, client secret, connection string)
API_KEY = "sk_live_1234567890abcdef" # fake example
CLIENT_SECRET = "super-secret-value" # fake example
SQL_CONN_STR = "Server=tcp:prod-sql.database.windows.net;User ID=admin;Password=P@ssw0rd!;" # fake example
logging.basicConfig(level=logging.INFO)
# Secret printed to logs
logging.info(f"Using API_KEY={API_KEY}")
# Potential PHI/PII printed
patient = {"name": "John Doe", "dob": "1985-01-01", "mrn": "123456"} # fake example
print("Patient record:", patient)
# Exfil risk: outbound call with sensitive header
r = requests.get(
"https://example-external-service.com/api/data",
headers={"Authorization": f"Bearer {API_KEY}"}
)
print(r.status_code)
# Writing secrets to storage (bad)
path = "/lakehouse/default/Files/debug/secrets_dump.txt"
with open(path, "w") as f:
f.write(f"{API_KEY}\n{CLIENT_SECRET}\n{SQL_CONN_STR}\n")
>>>
Response:
[
{
"policy_id": 1,
"evaluation": "red",
"details": "Hardcoded API key found: API_KEY = \"sk_live_1234567890abcdef\". The key is also printed to logs (logging.info(f\"Using API_KEY={API_KEY}\")) and sent in an outbound request Authorization header (headers={\"Authorization\": f\"Bearer {API_KEY}\"}). Additionally, the API key is written to a file (secrets_dump.txt)."
},
{
"policy_id": 2,
"evaluation": "red",
"details": "Hardcoded credentials found: CLIENT_SECRET = \"super-secret-value\". Hardcoded SQL connection string includes credentials and a password: SQL_CONN_STR = \"Server=...;User ID=admin;Password=P@ssw0rd!;\". These secrets are also written to storage (secrets_dump.txt)."
}
]Link to the notebook written by me, a Notebook content getter:
https://github.com/dataassets1/fabric_extension_sec_audit/blob/main/GetNotebooksContent_public.py
The link to the Demo of the prototype in Figma: https://sec-audit-fabric.figma.site/
(disclaimer: it is an emulated prototype; it is not a real Microsoft Fabric)
7 Further Development of the Solution
That solution demonstrates the possibilities of the automated audit in Fabric. Its current limitations: it shows the concept without its full implementation (even though the critical blocks of the solution were tested, such as AI abilities to recognize patterns and API abilities to get the Fabric content).
Further development may include:
1) Focus on the local and pretrained LLM to build a reliable, fast, and cheap LLM for pattern recognition in Microsoft Foundry.
2) Add more specific HIPAA- and other policies-related rules, to identify them.
3) Build a data pipeline to have convenient reporting and historical comparison.
3) Collaborate with stakeholders to identify the other gaps that may be solved.
8 Resources
Video Demo: https://www.youtube.com/watch?v=VVpQDq1sd4o
BR, Iurii (Yurri) Iurchenko
E-mail: 4iurchenko@gmail.com
LinkedIn: https://www.linkedin.com/in/4iurchenko/
