Forum Discussion
Capacity Derive
Hi BalajiL,
This is a pretty massive load. I think you'd need multiple capacities to handle this, as a single F2048 might not be enough for that much data volume happening that frequently.
Remember that the wya capacity throttling and smoothing works for background operations (like pipeline runs) is that the usage is smoothed over a 24 hour period, so running something 30 times over the day and running it 30 times all at once has the same impact to the capacity.
Are you able to provide more details? YOu mention that 33TB is the total data size, but do you know how much data is loaded generally for each batch?
How many tables are the batches loading?
Thanks tayloramy for your prompt response. Yet to know how much data is loaded in each batch ( am working on it) but the batch is running through out a day for six regions like (US, UK, APAC etc..) and multiple frequencies like 24 hr pipeline window, 2 hr pipeline, 4 hr pipeline window.
Hi BalajiL,
It sounds like you will need a lot of capacities.
I'd recommend starting with an F512 or F1024 and see how efficient you can build the processes, but be prepared to upgrade to F2048 and to purchase an additional capacity if required.If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.