Resource Governing error during dataset refresh in Fabric workspace

[TungDo_DEEPC]

I'm getting a ResourceGoverning error when refreshing datasets, which are in a workspace that I've assigned Fabric F2 SKU to very recently. This workspace was previously on a Pro plan, and the dataset in trouble is not using a Fabric lakehouse as the data source. It connects to Dataverse table and is designed to do quite a lot of data transformation on top of that data. Below is the error details:

Data source error: Resource Governing: This operation was canceled because there wasn't enough memory to finish running it. Either reduce the memory footprint of your dataset by doing things such as limiting the amount of imported data, or if using Power BI Premium, increase the memory of the Premium capacity where this dataset is hosted. More details: consumed memory 3115 MB, memory limit 3057 MB, database size before command execution 14 MB. See https://go.microsoft.com/fwlink/?linkid=2159753 to learn more.
Cluster URI: WABI-SOUTH-EAST-ASIA-redirect.analysis.windows.net
Activity ID: 00000000-0000-0000-0000-000000000000
Request ID: 71591015-8c62-4aea-883a-4fca4c09f18a
Time: 2024-12-20 00:43:42Z

I'd like to get community insight on few questions:

1. To immediate counter this issue, I plan to revert the workspace to Pro and keep the semantic models there. Other Fabric items (data pipeline, pySpark notebook, lakehouses) will be deploy to another workspace  which I will assign F2. Would this be a sound plan or do you think there is other optimal ways?
2. I suppose the low computation power with F2 is the cause of this issue. But as the error has never occured prior to making the switch to F2, does it mean the Pro plan actually provide MORE computing power than F2? If so, it is quite confusing to me. What is the reason behind F2's failing to handle a 14MB dataset with 3000MB memory footprint (this is not a high computation demand, isnt it?)
3. I can consider upgrading to F8 (going higher does not makes sense for us yet economically). Would this solve this error for us in long term? (in long term, I dont expect our datasets will grow much in term of storage and memory footprint)

[v-tsaipranay · Accepted solution]

Hi @TungDo_DEEPC ,

Thank you for reaching out to Microsoft Community Forum with your questions about the Resource Governing error you're encountering.

As per your question1: Moving your semantic models to Power BI Pro and using the F2 SKU for other Fabric tasks like data pipelines, notebooks, and Lakehouse’s is a smart and effective strategy. This approach will help you balance cost and performance, keep the Power BI Pro workspace optimized for your heavy reporting and analytics tasks, and take advantage of F2’s lower cost for simpler data engineering activities.

As your data grows, it's important to monitor the performance of your F2 workspace. If your data pipelines or transformations become more complex in the future, you might need to scale up to a higher SKU (like F3 or F8) to ensure everything continues to run smoothly.

For further scaling SKUs, use the below link for reference:

https://learn.microsoft.com/en-us/fabric/enterprise/scale-capacity

Question 2: The issue you're facing is because the F2 SKU has limited memory and compute resources, which can't handle the 3000MB memory footprint created by the transformations on your 14MB dataset. On the other hand, Power BI Pro offers more memory (up to 16GB) and is optimized for in-memory computations. This makes it better suited for complex transformations and large data models, which is why it was able to handle the dataset without any issues.

The F2 SKU is more suited for lightweight data engineering tasks and isn't equipped to handle large, memory-intensive data transformations that Power BI Pro is built for.

If you need more insights about capacities and their SKUs refer below link:

https://learn.microsoft.com/en-us/fabric/enterprise/licenses#capacity

Question 3: Upgrading to F8 would fix the issue and give you plenty of extra resources for the future, but it comes at a higher cost. If you're concerned about cost and don't expect your data to grow much, I’d suggest looking into F3 or F4. These options should be more affordable and still handle your current needs without extra capacity you don't really need.

I believe the details provided above have addressed your concerns. If you have any further questions or need additional information, please refer to the link below for more details:

Solved: Re: Resource Governing Error trying to refresh a D... - Microsoft Fabric Community

If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Thank you.

[TungDo_DEEPC]

@v-tsaipranay thanks for the reply. Things are much clearer for me now.

I already have a reason for upgrading to F8: F2 usually can accomodate 1 notebook run at the same time. (It usually blocks me while developing a notebook if at the same time another notebook is ran in a scheduled data pipeline refresh.)

If I may ask further here, is there an official documentation somewhere that I can refer to if I need to tell exactly the maximum memory footprint of a semantic model transformation that each Fabric SKU tier can accomodate?