Author: Matt Basile - Senior Product Manager
While OneLake makes it easy to grow your data estate, keeping long-term data can drive up costs over time. Admins need ways to control their storage costs without giving up historical data often required for compliance and auditing. With the introduction of OneLake storage tiers and lifecycle management, it's never been easier to optimize your storage bill on OneLake by moving your historical data into cost-effective tiers automatically.
Introducing storage tiers in OneLake
OneLake now supports three storage tiers:
- Hot tier – optimized for frequently accessed, active data.
- Cool tier – optimized for less frequently accessed data where lower storage cost is preferred.
- Cold tier – optimized for long-term retention and the lowest storage cost.
All data in OneLake is currently in the hot tier. The cool and cold tiers have higher transaction costs than the hot tier, and additional data retrieval charges, but lower storage costs. You can change a file’s tier directly, upon upload, by changing the workspace’s default tier, or automatically through a lifecycle management policy.
Automate data movement with lifecycle management
OneLake lifecycle management policies simplify tiering by automatically moving files between tiers with customizable rules. Rules can change tiers based on when a file was created, last modified, or last accessed. Once defined, policies run automatically in the background, making tier management simple and automatic!
For example, you can create policies such as:
- Move files that have not been modified in 30 days to the cool tier, and that have not been modified in 90 days to the cold tier.
- Move files within a specific path that have not been accessed in 30 days to the cool tier, and back to the hot tier when accessed.
This policy-driven approach and flexible conditions ensure a file's tier is always matched to its access pattern and your organization's data strategy.
Optimizing costs with storage tiers
Let’s review how storage tiers and lifecycle management can save your organization with an example. I’m a workspace admin with 10TB of bronze data in my workspace. I need to store this data for at least five years, and once per year, I perform a compliance audit which reads approximately 10% (1TB) of data in a day.
To start, calculate the cost of storing the data at each tier:
Tier | Storage Cost (per month) | Monthly storage cost | Total storage cost (60 months) |
Hot | $0.023 per GB | $230/month | $13,800 |
Cool | $0.0125 per GB | $125/month | $7,500 |
Cold | $.004 | $40/month | $2,400 |
By moving data to the cool or cold tier, I can see significant savings on my storage costs. But the cool and cold tiers also come with higher capacity consumption (CUs). Next, I’ll calculate the CU consumption for my yearly read at each tier, considering both the read transactions and data retrieval fee (for cool/cold tier). I'll also calculate what percent of my F64 capacity is utilized by my 1TB read.
Tier | Total CU consumption | Daily % of F64 Capacity |
Hot | 2,600 CU seconds | 0.0004% |
Cool | 206,500 CU seconds | 4% |
Cold | 665,000 CU seconds | 12% |
As shown in the previous table, accessing large amounts of data in the cool or cold tiers can increase your capacity consumption significantly. We recommend estimating your consumption and sizing your capacity appropriately. A temporary increase in capacity size is not likely to outweigh the amount you save in storage costs from keeping your data in the cool or cold tier!
Getting started
To get started with storage tiers and lifecycle management:
- Open the workspace in Microsoft Fabric (you must be a workspace admin).
- Go to Workspace settings > OneLake > Lifecycle management.
- Create a default lifecycle rule with pre-defined scope, actions, and conditions.
You can also use the new OneLake Lifecycle Management APIs or change tiers directly with the
With storage tiers and lifecycle management in OneLake, you now have even more control over managing your storage costs. To learn more, refer to the OneLake storage tiers and OneLake lifecycle management documentation.