Skip to main content
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Did you hear? There's a new SQL AI Developer certification (DP-800). Start preparing now and be one of the first to get certified. Register now

spellicerKUSA

control rate of lakehouse Materialized Lake View (MLV) when refreshing on schedule

Submitted by on ‎08-06-2025 05:42 AM

Currently when trying the new feature of refreshing MLVs on a schedule, on even a single run of the refresh, I get a random mix of successful and failed and the error for the failed is:

Error Code MLV_SPARK_JOB_CAPACITY_THROTTLING
Message To run, cancel an active Spark job through the Monitoring hub, choose a larger capacity SKU, or try again later. HTTP status code: 430

Slowing down or allowing for a retry would help for those of us with smaller capacities that can't handle the simultaneous spark job submissions.

Status: New
See more ideas labeled with:
4 Comments (4 New)
Comments
sairamyeturi
Microsoft Employee
Microsoft Employee
‎08-10-2025 10:02 PM
Thanks for the feedback, Can you also share the capacity size you use, and how many concurrent operations you run at the same time?
sebituranily
New Member
‎08-26-2025 12:17 AM
We’re seeing the same issue right now. Setup: F8 capacity, 1 lakehouse, 13 MLVs in the silver layer — all silver MLVs are running in parallel. If that’s the root cause, it would be great to have either a high-concurrency session for refreshing MLVs or an option to queue them automatically within the Spark limitations. On F8, the Spark concurrency limit kicks in and causes 8 MLVs to fail. When I refresh the MLVs manually via a separate notebook triggered by a pipeline, it works fine and is even faster than the lakehouse scheduler on FT1 (Trial). Key point: respect the lineage and refresh in the right order. REFRESH MATERIALIZED LAKE VIEW lakehouse.silver.xyz1; REFRESH MATERIALIZED LAKE VIEW lakehouse.silver.xyz2; [...] Would be great if the refresh could happen within Spark session limits across multiple workspaces.
spellicerKUSA
Regular Visitor
‎08-26-2025 12:50 PM
I ended up working around it with an airflow job submitting some spark queries through livy. It's really a lot more hassle than being able to turn on the automatic refresh. I use a pool to limit how many statements I'm sending at once, but it's all done in a single session so I don't run into the session exhaustion.
spellicerKUSA
Regular Visitor
‎11-17-2025 06:06 AM
This seems to be resolved in the latest round of improvements to the MLV refresh system