Forum Discussion
How to avoid people creating multiple times the same table in different workspaces
Dears,
We have a situation in Fabric where we have more than 6K workspaces.. Which seems to be an indicator that things are not properly governed. Hence we are starting a journey to govern it
1) People can only create workspaces at request, by filling in a template (which has a lot of information request, like which sensitive ifnromation will be added, what is the goal of the workspace, etc..)
2) We added Mcirosoft Purview to create our data products
But we still see in many projects teams, once they have their workspaces assigned , they start to create tables that exist already. For example, product table, or cusrtomer table, or others.. Specially, if purview does not have those yet formed as data products
I have requested project teams to start using the below fabric badges in production to see if it helps and we need to designate which people can add this badges
| Promoted | Data Engineer, Data Analyst, BI Developer, Report Author |
| Certified | Data Steward, Data Owner, Governance Team, Fabric/Power BI Admin |
| Master Data | Data Owner, Data Steward, Enterprise Data Governance Team |
But I believe this will not be enough in terms of tables duplication. We need re-usability not duplication..
What are you doing to secure that your project teams do not create duplicated tables?
currently we have 17M data assets in fabric and more than 6K workspaces.. this is becoming a problem and the exact problem we wish to solve is:
1) How can people see if a table already exists in Fabric (as many, may not want to go to purview or even if they want, purview may not yet have the data product formed). Are yo9u asking them to go to the fabric catalog?
2) How can we prevent them (if this is ven possible) from creating tables , if, for example, a customer table already exists in another or even on the same workspace, for example in different lakehouses?
What are you adopting to prevent this?
Thanks a lot,
Pedro
Hi fabricpribeiro ,
Thanks for the context — that helps.
With a mix of item types and domains already in place, you're in a good spot. Domains let you delegate certification and duplicate-review to domain admins, which is usually where central governance stalls at scale.
On Fabric APIs vs Purview — they solve different problems, not either/or:- Purview Unified Catalog = business-facing surface (glossary, data products, classifications, lineage across sources). Best for stewards and consumers.
- Fabric Catalog Search + admin APIs = programmatic/operational surface. Query Fabric metadata directly (items, workspaces, domains, endorsement, owners, schemas) to build custom duplicate-detection jobs, dashboards, and Teams alerts.
Purview for humans, Fabric APIs for machines. Purview scans are scheduled and great for the catalog view, but not built for real-time duplicate prevention. Fabric APIs let you run lightweight jobs that flag suspicious items as they appear.
APIs worth looking at:Microsoft Fabric REST API documentation
full-tenant item enumerationSImple Starting recipe: scanner API nightly → load into a governance Lakehouse → flag duplicate/near-duplicate table names outside certified workspaces → notify the domain steward. Cheap to build, high ROI at your scale.
Hope That helps!
7 Replies
- Parchitect
Solution Sage
This is not only a Fabric technical issue. It is a data governance and data architecture issue.Since you already have a workspace onboarding process, I would extend that process with a required data discovery and architecture review step.Before a team creates new tables, they should answer:- Does this data already exist in OneLake Catalog or Purview?- Is there already a certified/master data product for this entity?- Is this table a new domain-specific table, or a duplicate of an enterprise entity such as Customer or Product?- Should the team create a new table, or consume the existing one through a shortcut, shared Lakehouse/Warehouse, or shared semantic model?- Who owns the data product and who is allowed to certify or modify it?The architecture pattern I would use is:Authoritative/master data workspace→ Certified Customer/Product/Supplier tables→ Reuse through OneLake shortcuts or shared semantic models→ Domain/project workspaces consume trusted data instead of copying itOneLake Catalog should be the first discovery point because it is designed to help users find, explore, use, and govern Fabric items across workspaces. Purview should complement this with business glossary, lineage, sensitivity, ownership, and data product governance.Shortcuts are also important because they allow teams to reuse existing data without creating unnecessary copies.So the onboarding process should not only ask “why do you need a workspace?”It should also ask “what data do you plan to create, and does it already exist?”That is how you avoid turning Fabric into another spaghetti architecture. - Omkar_1712
Solution Specialist
Hello fabricpribeiro,
This is a common challenge as Fabric adoption grows. In my opinion, endorsements alone won't prevent duplication - they only help users identify trusted assets.
A few practices that have worked well are:
1. Make discovery the first step
- Encourage teams to search the OneLake Catalog (and Purview where available) before creating new tables. The easier it is to discover existing assets, the more likely they are to be reused.
2. Define authoritative data products
- Identify core entities such as Customer, Product, Supplier, etc., and designate a single Certified or Master Data source for each. Project teams should consume these instead of recreating them.
3. Promote reuse through architecture
- Instead of copying tables into every workspace, encourage reuse via OneLake shortcuts, shared Lakehouses/Warehouses, or shared Semantic Models where appropriate.
4. Establish governance gates
- As part of the workspace provisioning process, require teams to verify whether an equivalent data asset already exists before creating a new one. This can be included in your workspace request checklist.
5. Monitor and review duplication
- Use Purview and Fabric metadata to periodically identify duplicate assets and work with project teams to consolidate them where possible.
Regarding your specific questions:
1. How can people know whether a table already exists?
Today, the OneLake Catalog is probably the best starting point for discovery within Fabric, with Purview complementing it through richer governance and lineage.
2. Can Fabric prevent duplicate table creation?
As far as I'm aware, there isn't currently a built-in policy that prevents users from creating a table simply because one with the same business meaning already exists elsewhere. This is primarily addressed through governance, discoverability, and reuse patterns rather than technical enforcement.
With an environment of 6,000+ workspaces and 17 million data assets, I'd focus on making trusted data products easy to find and reuse, while introducing governance processes that discourage unnecessary duplication.
Best regards,
Omkar Shinde
Microsoft Fabric Enthusiast | Power BI Consultant
💡 If you found this response helpful, please consider giving it a Kudos.
✅ If this resolves your question, please mark it as the Accepted Solution to help others in the community. - Dev_Dholakia
Resolver IV
Hi fabricpribeiro ,
Honest answer at your scale: no single control stops duplicate tables. It has to be discovery + standards + light review.
Point everyone at the OneLake catalog as the mandatory first stop — search by name, schema, owner, badge, domain across all workspaces. Pair it with the Purview Unified Catalog for business glossary, data products, and lineage.
Fabric won't auto-block a CREATE TABLE because something similar exists elsewhere. Prevention comes from governance, not the engine — restricted write access on gold/master lakehouses, workspace templates, naming standards, and periodic duplication audits via admin APIs or Purview scans.Practical pattern that actually reduces duplication:
- Gold/master workspaces for shared entities (Customer, Product) with write access locked to a small stewardship group. Others consume via OneLake shortcuts, not by re-creating tables.
- Endorsement badges (Promoted / Certified / Master data) for trust and search ranking — helpful, but signals only, not enforcement.
- Domains + naming standards so authoritative tables live in a known place with predictable names; delegate certification to domain admins.
- Purview data products for the business view — start with your top 10–20 entities even if the model isn't complete.
- Workspace request process (which you already do) plus a "does this already exist?" checklist referencing the catalog.
- Monitoring with the Catalog Search REST API or admin APIs to flag likely duplicates for stewards.
Short version: OneLake catalog + Purview for discovery, shortcuts to certified gold for reuse, a small steward group owning master entities. Badges and templates support this — they don't replace it.
Two quick clarifications:
Are the 17M assets mostly Lakehouse tables, or a mix (warehouses, semantic models, shortcuts)?
Do you already have Fabric domains defined, or is workspace-by-request your only organizing layer today?
If this got you what you needed, a Kudos and an Accepted Solution mark would be great — it helps others searching for the same thing find the answer quicker.- fabricpribeiro
Post Patron
"Are the 17M assets mostly Lakehouse tables, or a mix (warehouses, semantic models, shortcuts)" Its a mix
"
Do you already have Fabric domains defined, or is workspace-by-request your only organizing layer today?
" We have Domains in Fabric
"Monitoring with the Catalog Search REST API or admin APIs " What is the advantage of using this instead of discovery via Purview? and which APIs are these ?
Thanks a lot
- Dev_Dholakia
Resolver IV
Hi fabricpribeiro ,
Thanks for the context — that helps.
With a mix of item types and domains already in place, you're in a good spot. Domains let you delegate certification and duplicate-review to domain admins, which is usually where central governance stalls at scale.
On Fabric APIs vs Purview — they solve different problems, not either/or:- Purview Unified Catalog = business-facing surface (glossary, data products, classifications, lineage across sources). Best for stewards and consumers.
- Fabric Catalog Search + admin APIs = programmatic/operational surface. Query Fabric metadata directly (items, workspaces, domains, endorsement, owners, schemas) to build custom duplicate-detection jobs, dashboards, and Teams alerts.
Purview for humans, Fabric APIs for machines. Purview scans are scheduled and great for the catalog view, but not built for real-time duplicate prevention. Fabric APIs let you run lightweight jobs that flag suspicious items as they appear.
APIs worth looking at:Microsoft Fabric REST API documentation
full-tenant item enumerationSImple Starting recipe: scanner API nightly → load into a governance Lakehouse → flag duplicate/near-duplicate table names outside certified workspaces → notify the domain steward. Cheap to build, high ROI at your scale.
Hope That helps!
- v-sathmakuri
Community Support
Hi fabricpribeiro ,
Could you please check the provided solution and let us know if you have any further questions.
Thanks!!
- v-sathmakuri
Community Support
Hi fabricpribeiro ,
Could you review the suggestion provided above and let us know if you have any additional questions, we are happy to address.
Thanks!!