Forum Discussion
A problem has occurred accessing your storage account or content
- 2 years ago
Hi amolt !
Could you please share your ticket ID / service request ID with me so I can take a closer look at your case? You can post it here or send it privately to me over a direct message. I wonder what could be consuming so much and if perhaps there are opportunities of improvements at the query level.
I will be marking this as resolved as the underlying issue is related to capacity and just resource consumption. Such resource consumptions depend on a number of factors beyond just the capacity as it also relies on your data source, the connector in use and the overall query or dataflow that was created. There are techniques in which you could stage the data first and perform the transformations afterwards, so we could privately explore those options that would work best for your specific case.
Best!
Hi,
Thanks for your feedback.
Although my capacity (F4) has now been paused for two days, I am still unable to generate a 19-line PDF report without triggering an error: "The visual element has exceeded the available resources". If even such a simple action isn't possible with an F4 capability, I wonder what the point of sizing is. It's useless.
I agree there is a definite issue here!
I am processing a total of 4 Gb of data, using a combination of dataflows (Gen2), pipelines and Notebooks - with 3 lakehouses and 1 warehouse.
I am using an F8 capacity and after 24 hour on my new capacity, I am back on the same situation. I had to revert to the Trial capacity again.
If I really need a F64 capacity to process 4 GB of data, Fabric may not be a feasable option for us.
- amolt2 years ago
Resolver I
Hi Anonymous ,
In fact I had to do the same thing as you, I went back to trial capacity.
However, I noticed that the smaller my Gen2 dataflows were, the more likely they were to terminate. So I have a set of dataflows that I run in series and certainly not in parallel. But despite this, I still have problems with dataflows that merge two requests and then the process never finishes correctly.
If possible, you could try reducing the size of your dataflows and serializing their execution in a pipeline. I hope this helps.- Anonymous2 years agoNot applicable
HAHA! This is exactly what I was in the process of doing/trying.
FYI: the only reason I am using dataflows is to access data through the on-prem data gateway to our on-prem SQL servers, which is not currently possible with the Copy Data activity... So since I am not doing any merges, I am hoping this will help stabilize my pipelines.
Let's keep this conversation going! Thank you for sharing your experience with me. 🙂
- Anonymous2 years agoNot applicable
This is my current stats (if it can help anyone):
Utilization:
Overages: