The Law of The Minimum
Long before computers, two chemists named Sprengel and Liebig were working in the area of agricultural chemistry. Among their many accomplishments, Sprengel pioneered and Liebig popularized the Law of the Minimum, which states that growth is limited by the least available resource. If a plant is starving for nitrogen, then only additional nitrogen will get things growing again. Everything else you give it doesn’t help. Oddly enough the Law of the Minimum applies to computers, too.
When you are out of X, adding anything else but X won’t help at all. Extra resources may help new transactions move through the system quicker, but all that really does is hurry them to the X bottleneck, where they will find themselves at the end of a very long line of transactions waiting their turn. If you screw up and add the wrong resource, the queue of waiting tasks at the least available resource may just get longer and there will be no positive effect on throughput or response time.
In the biological sciences too much of something (water, warmth, etc.) can kill you just as easily as too little. In computers, if you have too much of some resource there is no negative effect on the overall capacity of the system to get things done. Every so often I run into people who believe the myth that a system has too much of a given resource and that somehow hurts performance. It takes considerable time to talk them out of this belief. Be patient with them. It is the case that when something suddenly and rapidly dumps work into the system (e.g. a system coming back online after a comm failure) then that can cause a performance disruption. But the right fix for that is to put a bit of flow control in so these rare events don’t drown the system in transactions and kill performance.
In business, money is often the most limited resource. If the system has too much of some resource, you are wasting money. The trick is to always have just enough resources in place to handle the peak plus a bit more as a margin of safety. Any fool can solve bottlenecks with limitless money.
The Hidden Bottleneck
When you run out of some resource, that resource becomes a bottleneck. All the transactions race through the system only to find a huge queue because of that resource limitation.
The double-necked hourglass illustration shows a bottleneck at point A. Beyond that bottleneck life is easy for the rest of the system as, no matter how many transactions arrive, the workload is throttled by the upstream bottleneck.
If you “fix” bottleneck A then performance will be really good for only about 45 milliseconds until that great load of transactions hits bottleneck B with a sickening “WHUMP!” The throughput of this system will hardly change at all, and you will have some explaining to do in the boardroom.
When capacity planning, it is important to explain this drawing to the decision makers so they comprehend how one bottleneck can hide a downstream bottleneck. It is also key for you to meter all the resources you can deplete, not just the one that is the obvious bottleneck.
If your response time is growing, then there is a bottleneck somewhere. If the meters you are looking at (for bottleneck B) are showing lots of capacity, then you are looking in the wrong place. However, it is important to look at them as they can tell about your future. If you need this hourglass to do 10X the work it is doing now, and the meters for bottleneck B are showing that part of the system is 50% busy, then there is no way that part of the system can do 10x the work. Your current problem may lie elsewhere, but you’d better put bottleneck B on your to-do list.
The hourglass could easily have been drawn with many more bottlenecks, but I’ve never seen a performance problem where there were more than two bottlenecks that had to be cleared up to get the needed throughput. If you are working on the fourth bottleneck for this given problem, then perhaps you should spend some time thinking about a new career – because you are most likely deep in the weeds.
For more information on finding, fixing, and avoiding bottlenecks, as well as capacity planning, I’d suggest you read my book The Every Computer Performance Book
For more details on Liebig’s Law of the Minimum see: http://en.wikipedia.org/wiki/Liebig%27s_law_of_the_minimum