Forum Discussion
Performance Difference Between Fabric Shortcuts and Synapse Serverless SQL Pool
Hello everyone,
I am currently evaluating different ways to query data stored in Azure Data Lake Storage Gen2 and noticed a significant performance difference that I would like to better understand.
My current architecture uses Synapse Serverless SQL Pool accessed through a Node.js application (Tedious driver). Recently, I created a Shortcut in Microsoft Fabric pointing to the same ADLS data and started running equivalent queries through Fabric.
To my surprise, queries executed through Fabric appear to be considerably faster than those executed through Synapse Serverless, even though both are accessing the same underlying data.
Environment:
- Azure Data Lake Storage Gen2
- Microsoft Fabric Shortcut
- Synapse Serverless SQL Pool
- Delta Lake datasets
- Analytical queries with filters and pagination
- Datasets containing millions of records
- Same Azure region for storage and compute
My questions are:
- What optimizations does Fabric apply when querying data through Shortcuts?
- Does Fabric leverage metadata caching, data caching, or other performance enhancements that are not available in Synapse Serverless?
- Has anyone else observed similar performance differences when comparing Fabric and Synapse Serverless against the same ADLS data?
Any insights, documentation, or real-world experiences would be greatly appreciated.
Thank you!
Hi luanlopes,
Hope you're doing well!
In my opinion and from my experience, I can say Yes, Fabric applies fundamental architectural optimizations that Synapse Serverless simply was not designed to provide. Synapse Serverless was purpose-built as a cost-efficient, cold, ad-hoc query engine. Fabric is built as a performance-first analytical platform with caching at every layer (metadata, data, results), a native Delta Lake engine, warm compute, and an internal OneLake network path. The performance difference you observed is expected, reproducible, and well-founded architecturally.
Thanks for the explanation.
Do you have more details about the caching layers you mentioned?
Specifically, do you know which cache layers apply when querying ADLS Gen2 data through OneLake shortcuts using SQL Analytics Endpoint or GraphQL API?
I found Microsoft docs mentioning shortcut caching for S3/GCS/on-prem sources, and intelligent cache for Spark, but I couldn’t find anything explicit for ADLS Gen2 shortcuts with SQL Endpoint/GraphQL.
Do you know if Fabric applies metadata cache, data cache, or result cache in this scenario? And is there any official documentation or architecture diagram about it?
Hi luanlopes , let's tackle one question a time:
#1. What optimizations does Fabric apply when querying data through Shortcuts?
Fabric's SQL analytics endpoint and Lakehouse are powered by a modern engine optimized specifically for running T-SQL queries over Delta Lake data in OneLake whereas Synapse Serverless uses an older distributed query processing engine designed for a broader range of storage formats, not purpose-built for Delta
#2. Does Fabric leverage metadata caching, data caching, or other performance enhancements that are not available in Synapse Serverless?
Yes! Lots more , just to mention the most important related to your question are the v-Order optimization (a write-time optimization applied to Parquet files that improves read performance), delta log metadata caching, auto transaction log checkpointing, auto statistics for Delta tables, among other enhacements 😉
#3. Has anyone else observed similar performance differences when comparing Fabric and Synapse Serverless against the same ADLS data?
Yes, one of the main reasons I just completly deprecated Synapse in favor or Fabric ... remember, Synpse will eventually go away as Microsoft is shifting its major development and investment efforts toward Microsoft Fabric.
So, I would suggest you to, if possible, try to switch early in the game 😉
Hope you find this post answers your questions, if so, mark as solution... a thumbs up would also be nice 😊
All the very best.
3 Replies
Hi luanlopes,
Hope you're doing well!
In my opinion and from my experience, I can say Yes, Fabric applies fundamental architectural optimizations that Synapse Serverless simply was not designed to provide. Synapse Serverless was purpose-built as a cost-efficient, cold, ad-hoc query engine. Fabric is built as a performance-first analytical platform with caching at every layer (metadata, data, results), a native Delta Lake engine, warm compute, and an internal OneLake network path. The performance difference you observed is expected, reproducible, and well-founded architecturally.
Thanks for the explanation.
Do you have more details about the caching layers you mentioned?
Specifically, do you know which cache layers apply when querying ADLS Gen2 data through OneLake shortcuts using SQL Analytics Endpoint or GraphQL API?
I found Microsoft docs mentioning shortcut caching for S3/GCS/on-prem sources, and intelligent cache for Spark, but I couldn’t find anything explicit for ADLS Gen2 shortcuts with SQL Endpoint/GraphQL.
Do you know if Fabric applies metadata cache, data cache, or result cache in this scenario? And is there any official documentation or architecture diagram about it?
Hi luanlopes , let's tackle one question a time:
#1. What optimizations does Fabric apply when querying data through Shortcuts?
Fabric's SQL analytics endpoint and Lakehouse are powered by a modern engine optimized specifically for running T-SQL queries over Delta Lake data in OneLake whereas Synapse Serverless uses an older distributed query processing engine designed for a broader range of storage formats, not purpose-built for Delta
#2. Does Fabric leverage metadata caching, data caching, or other performance enhancements that are not available in Synapse Serverless?
Yes! Lots more , just to mention the most important related to your question are the v-Order optimization (a write-time optimization applied to Parquet files that improves read performance), delta log metadata caching, auto transaction log checkpointing, auto statistics for Delta tables, among other enhacements 😉
#3. Has anyone else observed similar performance differences when comparing Fabric and Synapse Serverless against the same ADLS data?
Yes, one of the main reasons I just completly deprecated Synapse in favor or Fabric ... remember, Synpse will eventually go away as Microsoft is shifting its major development and investment efforts toward Microsoft Fabric.
So, I would suggest you to, if possible, try to switch early in the game 😉
Hope you find this post answers your questions, if so, mark as solution... a thumbs up would also be nice 😊
All the very best.