Forum Discussion
User Configuration Issue - Fail to Update Tables in Pipeline
Hi vwynn23
Yes, your options are to either reschedule the pipeline or increase capacity. Fabric doesn’t currently support automatic or time-based scaling (e.g., scale up for 2 hours and down again), so any capacity changes would need to be done manually via the Admin Portal or through PowerShell/REST API scripts.
If rescheduling works reliably, that’s usually the simpler option. But if large files are a regular load, increasing capacity could provide more consistent performance.
If this solution helps, please consider giving us Kudos and accepting it as the solution so that it may assist other members in the community
Thank you.
Hi vwynn23
Thank you for reaching out microsoft fabric community forum.
Based on your description and the screenshots provided, it appears that the notebook consistently fails around 1:00 AM when processing the VIP file—likely due to its size—yet succeeds when rerun later in the morning. This behavior is commonly observed when capacity constraints or transient resource throttling occur during scheduled runs, especially on shared or lower-tier capacities (such as F SKUs). To mitigate this, we recommend implementing retry logic in your pipeline to automatically handle transient failures. You can also consider adjusting the pipeline schedule to a later time (e.g., 2:00–3:00 AM) when system load may be lower. Additionally, reviewing your workspace's capacity utilization and testing the pipeline on a higher or dedicated capacity can help confirm if the issue is resource-related. For further guidance, please refer to the Optimize Fabric Capacity documentation.
If this solution helps, please consider giving us Kudos and accepting it as the solution so that it may assist other members in the community
Thank you.
Thanks for your response, so my options are to either increase the capacity or have it later schedule? If we were to increase the capacity is it possible to scale the capacity up for 2 hours automatically and then downsize again?