Soft deletion for OneLake files makes sure that deleted files are restorable for 7 days. However, customers also have to pay for the storage usage during that time. For the scenario, when writing short-lived (large) temporary files to OneLake, this feature might have significant billing cost (see model calculation below). Scenarios, were deleting immediately would make sense: Executing Unittests and Performance tests, where the written data are only important temporarily When making use of Spark Checkpointing that will offload intermedite compute results to OneLake Request: give customers more control over that feature and let them descide, what is the right retention period for them, as well as disabling the feature all together for a Workspace, Lakehouse (or idealy for specifc folders). Cost comparison: Pricing page only states GB/month/GB-price (lets assume $0.023 for this model calculation). What happens when I do some performance testing (that takes 30 minutes) and write some large temporary files (50 TB total) to OneLake and delete it right after my tests? With Soft-deletion disabled, smalles billing period for Azure Storage Hot ist 1h; storing 50 TB for 1h would cost: $1.64 = $0.023 / 24 [hours] / 30 [days] * 1024 * 50 [TB] With Soft-deletion enabled, I'll need to pay for 1h + 7 days (retention period) for the 50 TB: $269.93 = $0.023 / 24 [hours] / 30 [days] * 1024 * 50 [TB] * (7 [days] * 24 [hours] + 1 [hour])
... View more