Simple and intuitive Warehouse experience

priyasa · ‎05-23-2023

See Arun Ulagaratchagan’s blog post to read the full Microsoft Fabric preview announcement.

In today's data-driven world, data warehousing is a critical component of any business strategy. Its goal is to assist companies in effectively managing and analyzing vast amounts of data, enabling them to make informed decisions and drive growth. However, with the ever-increasing amount of data being collected, the challenges associated with its management are also growing and this is compounded by the evolution of the era of AI. Traditional data warehousing solutions have become complex and costly, often resulting in data duplication, vendor lock-ins and governance issues.

We are thrilled to announce the preview of Synapse Data Warehouse in Microsoft Fabric! Synapse Data Warehouse is the next generation of data warehousing in Microsoft Fabric that is the first transactional data warehouse to natively support an open data format enabling IT teams, data engineers and business users to collaborate seamlessly and extract actionable insights from their data, all without compromising enterprise security or governance. Just like the previous data warehouse generation, SQL provides multi-table ACID transactional guarantees. It is built on the well-established SQL Server Query Optimizer and Distributed Query Processing engine but is bolstered with the key improvements below that add significant new value to enterprises -

- Fully managed: this new data warehouse is a fully managed SaaS solution and effortlessly extends modern data architectures to both professional developers who love to write code and citizen enthusiasts with no coding skills.  What previously took enterprises months to accomplish can now be done in minutes efficiently.
- No provisioning and managing of resources: instead of provisioning dedicated clusters, it is based on a fully serverless compute infrastructure where resources are provisioned in milliseconds as jobs requests come in. Enterprises benefit from resource efficiencies and only pay for what they use.
- Separation of storage and compute: compute nodes used are independent of storage enabling enterprises to scale and pay for either one separately.
- Open data standards: data is not locked-in the proprietary SQL Server format but is stored in the open data standard of Delta-Parquet in the Microsoft OneLake providing interoperability not only with all workloads in Fabric but also the Spark ecosystem without requiring any data movement.
- Cross-querying: as a result of the open data standard support, data in the lake whether processed by a Fabric workload or any other compute engine can be queried and cross-joined without making any copies of the data.
- Auto-scaling: it automatically scales resources instantly as query and usage requirements increase and down-scales when there is no more need for these resources, all without any user intervention.
- Self-optimizing: it automatically detects and isolates workloads to deliver predictable performance. The best performance is based on caching which is automatic and multi-tiered based on activity. Query plans generated are optimal. There is no need to hire highly skilled engineers to manage workload groups or tune the data warehouse.
- Fully integrated: it is fully integrated with all Fabric workloads right out of the box for any developer. Users can continue to benefit from the rich capabilities of the SQL engine using the T-SQL language or a simple user interface. All this with the continued benefits of the SQL ecosystem.

Let’s dive deeper into its capabilities.

Analytics is a continuum of activities requiring the ability to ingest, prep and analyze data, creating business semantics, machine learning models and BI reports. It requires collaboration across IT, data engineers, business analysts and data scientists across the organization. As data is shared or discovered, it needs to be secured and governed. Synapse Data Warehouse makes this easy through the key capabilities below.

Simple and intuitive Warehouse experience

Fabric has a dedicated home page for data warehouse, where a new Warehouse can be created with just a name and sensitivity label. There’s no provisioning or setup required. The user interface is a familiar relational database experience. The Warehouse Explorer exposes schemas, tables, stored procedures, and all other database objects. Anyone new to warehousing can start with a Warehouse sample!