Forum Discussion
Fabric Workspace Architecture
As the use case has limited 1 Data Engineer, I would recommend it to go with Option 3 as it will have :
- Very easy to manage
- Everything in one place
- Easier monitoring
- Less clicking around and
- Faster development.
But i will also have few drawbacks like, can become messy if naming standards are poor, less security control etc.
Once the project mature and team expand you can gradually shift towards option-2 or Option-1
Super UserHi reddyr2502,
This depends on your requirements. Is there a need for data seperation between layers (either for security reasons, or company policy?)
It also depends on how many items you have in the workspace, there is a limit to the number of items in a single workspace, though my opinion is that if you hit that limit you've done something wrong.
In my enviornment, I have all three layers in a single workspace. My bronze and silver layers are fully automated and controlled through metadata, where my gold layer is bespoke for each final table. This sounds similar to what you are trying to do, though I am doing it with notebooks and not with pipelines.
I would advise against creating a new workspace just for NetSuite as that will just create data silos and lead to challenges later when you want to integrate data from multiple sources into a single report.
Approach 2 can make sense if you need seperation between the layers, or if your design has hundreds of items per layer.
Approach 3 is the simpliest, and that's my vote.