Forum Discussion
Fabric Shortcuts and Cache
Hi tan_thiamhuat ,
That is an interesting question!
Shortcuts in OneLake allow you to quickly and easily source data from external cloud providers and use it across all Fabric workloads such as Power BI reports, SQL, Spark and Kusto. However, each time these workloads read data from cross-cloud sources, the source provider (AWS, GCP) charges additional egress fees on the data. Thankfully, shortcut caching allows the data to only be sourced once and then used across all Fabric workloads without additional egress fees.With the general availability for cross-cloud shortcut cache, new capabilities have been added as well. You now have the ability to define the retention period for your shortcut cache. Previously, data was only cached for 24hr, with these updates, you can select a retention period from 1-28 days. This greatly improves the effectiveness and cost savings of cache for sources that are not accessed every day.
Whereas intelligent cache optimizes Spark job performance by caching data at the Spark node level.It also automatically detects changes to the underlying files and automatically refreshes the files in the cache, providing you with the most recent data. When the cache size reaches its limit, the cache automatically releases the least read data to make space for more recent data. This feature lowers the total cost of ownership by improving performance up to 60% on subsequent reads of the files that are stored in the available cache.
This feature benefits you if:-
Your workload requires reading the same file multiple times and the file size fits in the cache.
-
Your workload uses Delta Lake tables, Parquet, or CSV file formats.
So no, they are not the same, though they are complementary.
Refer - Intelligent cache in Microsoft Fabric
Hope this helps!
-
Hi tan_thiamhuat ,
That is an interesting question!
Shortcuts in OneLake allow you to quickly and easily source data from external cloud providers and use it across all Fabric workloads such as Power BI reports, SQL, Spark and Kusto. However, each time these workloads read data from cross-cloud sources, the source provider (AWS, GCP) charges additional egress fees on the data. Thankfully, shortcut caching allows the data to only be sourced once and then used across all Fabric workloads without additional egress fees.With the general availability for cross-cloud shortcut cache, new capabilities have been added as well. You now have the ability to define the retention period for your shortcut cache. Previously, data was only cached for 24hr, with these updates, you can select a retention period from 1-28 days. This greatly improves the effectiveness and cost savings of cache for sources that are not accessed every day.
Whereas intelligent cache optimizes Spark job performance by caching data at the Spark node level.It also automatically detects changes to the underlying files and automatically refreshes the files in the cache, providing you with the most recent data. When the cache size reaches its limit, the cache automatically releases the least read data to make space for more recent data. This feature lowers the total cost of ownership by improving performance up to 60% on subsequent reads of the files that are stored in the available cache.
This feature benefits you if:
-
Your workload requires reading the same file multiple times and the file size fits in the cache.
-
Your workload uses Delta Lake tables, Parquet, or CSV file formats.
So no, they are not the same, though they are complementary.
Refer - Intelligent cache in Microsoft Fabric
Hope this helps!