topic Re: Microsoft Fabrics best practice architecture for Datawarehouse in Data Warehouse

Microsoft Fabrics best practice architecture for Datawarehouse

mshujaweb — Tue, 10 Sep 2024 09:32:13 GMT

Hi there,

Anyone aware of best practice Microsoft Fabric architecture, specially for datawarehouse experience, covering all best practices for availability, security, governance, connectivity, data flow etc.

Re: Microsoft Fabrics best practice architecture for Datawarehouse

Anonymous — Wed, 11 Sep 2024 02:54:50 GMT

Hi mshujaweb ,

Here are some summaries of data warehouse availability, security, governance, connectivity, data flow, etc:

Data storage: store data using Delta-parquet format for ACID transactions and interoperability without multiple copies of data.
Cross-database querying: use multiple data sources to gain insights quickly without duplicating data.
Data Connectivity: Connect to a variety of data sources by introducing, loading, and transforming data at scale through pipelines, data streams, cross-database queries, or COPY INTO commands.
Performance Management: Autonomous workload management using a distributed query processing engine ensures optimal performance.
Availability: Suitable for users of all skill levels.
Security: RBAC and the principle of least privilege are implemented to restrict access to sensitive data.
Data Management: Maintain metadata using a data catalog to ensure data discoverability.

If you have any other questions please feel free to contact me.

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

Re: Microsoft Fabrics best practice architecture for Datawarehouse

mshujaweb — Wed, 11 Sep 2024 05:52:14 GMT

Thanks Yang for your response and helpful tips on all those points. Are there any architecture diagrams as well, visio or otherwise, that we can refer to, for Microsoft Fabrics ?

Re: Microsoft Fabrics best practice architecture for Datawarehouse

vdgarcia — Wed, 11 Sep 2024 21:49:19 GMT

Hi, Yang,

In a scenario like the one at the company where I work, which decides to have an internal development team to use Microsoft Fabric and, through external companies, supply the needs that the internal team does not have the capacity to meet. What would be the best practices to configure a development, quality and production environment with third parties that allows version control and avoids as much as possible the constant re-configuration of connections for each environment?

Thank you very much for your attention.

Re: Microsoft Fabrics best practice architecture for Datawarehouse

Anonymous — Thu, 12 Sep 2024 07:35:27 GMT

Hi mshujaweb ,

Thanks for the reply from vdgarcia .

Here is the architecture diagram about fabric lakehouse and warehouse for your reference:

If you want more detailed information, please refer to the official documentation below:

Lakehouse end-to-end scenario: overview and architecture - Microsoft Fabric | Microsoft Learn

Data warehouse tutorial - introduction - Microsoft Fabric | Microsoft Learn

I found a blog that was helpful in delving into the use cases, differences and architectural design of Lakehouse vs Data Warehouse.

Lakehouse vs Data Warehouse vs Real-Time Analytics/KQL Database: Deep Dive into Use Cases, Differences, and Architecture Designs | Microsoft Fabric Blog | Microsoft Fabric

If you have any other questions please feel free to contact me.

Best Regards,
Yang
Community Support Team