Blog Post

Fabric Updates Blog
2 MIN READ

New metadata sync and more in SQL Analytics Endpoint (Preview)

rakrish75's avatar
rakrish75
Icon for Microsoft Employee rankMicrosoft Employee
3 months ago

Every Lakehouse and mirrored database in Microsoft Fabric automatically comes with a SQL Analytics Endpoint. It gives you a familiar T-SQL surface over the Delta tables in your lakehouse, so reporting tools, stored procedures, and ad-hoc SQL all just work. It is the bridge that lets the data you land in OneLake become instantly queryable by anyone who speaks SQL.

Two new updates are now available in preview, each addressing around data freshness and advanced capabilities on SQL analytics endpoint. Together, they expand what you can expect from your SQL analytics endpoint across data freshness, scale, and the types of analysis it can support.

Historically, getting the newly landed data in the lakehouse available for querying has been one of the more difficult aspects of our user experience. In response to customer feedback, the new metadata sync is now available in preview.

This new metadata sync features:

  • A brand-new architecture that is at parsing the delta logs and ensuring your data is available for querying with minimal delay.
  • Optimized background processes that decouple the data changes from schema changes, thereby ensuring more seamless data availability for different scenarios.
  • An on-query refresh of the tables to ensure you can read the data via sql analytics endpoint in near real-time.

Combined, these updates should help ensure your data is available for querying in seconds—not minutes—once its available on the lakehouse. For more details about the new metadata sync and how to enable this on your workspace, please check out learn more about SQL analytics endpoint.

Figure: Animated GIF - Workspace setting to enable the new metadata sync.

Time travel in SQL Analytics Endpoint

Time travel, one of the most useful features in data warehouse, is also available in preview for the SQL Analytics Endpoint.

This capability supports common scenarios data teams encounter every day: validating historical numbers for audits, investigating unexpected changes in downstream reports, or comparing metrics across points in time.

With time travel, you can query your data as if it existed at any point within the retention period configured in the lakehouse, without creating copies or maintaining separate historical datasets. For more details on time travel, refer to the Time Travel in Data Warehouse documentation.

Figure: Illustration of time travel in SQL analytics endpoint.

Get started

Both features are rolling out in preview with the new metadata sync. We would love to hear how you're using these capabilities. Share your feedback with us through the Fabric Community or your Microsoft account team.

Resources:

Updated 3 months ago
Version 1.0

5 Comments

  • rakrish75's avatar
    rakrish75
    Icon for Microsoft Employee rankMicrosoft Employee

    The UX setting is enabled through regions. It should show up in a few hours.

    Yes, this feature is now scoped to new workspaces. We are working on enabling a feature that allows customers to upgrade their existing endpoints. This feature will come later this year

  • The article that is linked to says "You can enable the new metadata sync process, which applies only to new SQL analytics endpoints." So this won't apply to any existing endpoints? Is that caveat just while it is in preview?

  • pbraun's avatar
    pbraun
    Regular Visitor

    When will this preview feature be available for capacities in West Europe? I currently don't have the option under Data Warehouse.

  • Lozovskyi's avatar
    Lozovskyi
    Icon for Kudo Collector rankKudo Collector

    Solid release. I really appreciate the automated metadata sync on query execution, as I previously had to orchestrate complex custom pipelines to ensure tables refreshed immediately after data insertion. Having native T-SQL time travel built directly into the SQL analytics endpoint saves me a significant amount of development time for historical audits.

    THANK YOU!

  • "With time travel, you can query your data as if it existed at any point within the retention period configured in the lakehouse, without creating copies or maintaining separate historical datasets."

    I'm hoping the work "if" above was accidentally included. It has a much different meaning with the word "if".

    I suspect that it relies on the fact that it did, indeed exist in another way in the past.