The Future of IT Infrastructure – Where Do We Go From Here?

This article was written by Steve Carl, Sr. Manager, Global Data Centers, Server Provisioning at BMC Software


In a series of articles I did in my Green IT blog at BMC, I documented the design we had chosen to consolidate not just our Data Centers, but all the server platforms therein. Collectively I called it “Go Big to Get Small

The thing I have been thinking about since then is “Whats next”? Two years ago when we set out to do this, we had to research, pick a design and commit to it, and to some degree hope that the technology stack we went with for each platform would be viable all during the course of the first phase of the project. Not that we could not adjust along that way, but standards simplify such complex, long term projects such as taking two data centers at over 40,000 square feet and making them one DC at 2,500 square feet. The fewer variables and moving parts, the easier it is.

At Dell World last week I heard someone talking about “Moore’s Law”.  They were blithely putting it out there as if it was the same thing  as a law of nature.  Its not of course, and it has not been served well by having the suffix of ‘Law’ attached to it. It is not the second law of thermodynamics. Its an observation of a trend.

A trend that is abating.
[subtitle3] Moore [/subtitle3]

Here is an example: two years ago we set the standard X86 computing node for virtualization as the Dell M620 blade, with 2 sockets, 16 cores, and 256 GB of RAM. That would, according to our performance and capacity planning tool, be on average a node that would run about 50 VMs of our average VM type. The CPU would average about 50% or so, and the RAM about 80% or so. Given how big the clusters of nodes were, that was extremely well performing, and also highly fault tolerant.

Two years later I ran the numbers and it turned out that exactly same configuration was STILL the sweet spot in price / performance. I was SURE that the answer would say that now would be a good time to move to higher core counts and denser memory. I was only going for a 50% increase in RAM, and a few more cores: 384 GB and 20 cores to be specific. It was doable, but the price was MORE than 50% higher, so it made no sense. I counted power, space, heat load, and the whole enchilada.  It may be different now with the new M630 being out: I have not re-run the number yet for it.

For the purposes of this post however, the main point is that Moore’s Observation has a finite end date to it. It will decelerate. Every turn of the technical crank will end up getting more expensive at some point.

We’ll still be able to shrink our Data Centers and use less power, etc, for a while yet. There are too many DC’s running around with ancient gear in them for that not to be true.
[subtitle3] Store [/subtitle3]

Its not just about the CPU or server side of it either. Physical, spinning disks (what we used to call the ’round and brown’, even though they are all silver these days as far as I know) are reaching what they can do with Areal Density. It was never on the same trajectory as Moore’s Observation, with this article noting a doubling in 5 years, not 18 months.

Most folks think spinning media is doomed soon anyway. Flash memory will sooner or later kill it for both power and density reasons.

Either way, same ultimate issue as a CPU then: Quantum mechanics are going to limit how small something can get. Once that happens, there is also the limit for power reductions and DC size reductions, not counting playing games with only powering up what’s in use, possible packaging innovations, etc.


[subtitle3] Virtualization [/subtitle3]

Virtualization has been a free ride for nearly 1.5 decades in the AMD64/X86 space, and since the early 1970?s on the mainframe. It may have fairly low overhead these days, but its ability to help consolidate starts to dry up once you move the average CPU / memory utilization up near the maximum possible. We had data centers FULL of barely used computers, with the average around something like 3% in our shop (much lower than the 10-15% often quoted out in the trades. Being an R&D shop, we had reasons why our servers were both sprawled, and lower utilization)

Once you have driven those numbers into the 80?s and 90?s you are starting to be out of road there though. The next great hope is technology like LXC / Docker / et al. Re-virtualizing the same OS over and over will eventually get old, and it will give way to application virtualization.. And then you’ll be at the wall again.


[subtitle3] Capacity Planning [/subtitle3]

The role of capacity planning just gets more and more central as the bottleneck game is played. Remove one barrier to full utilization and the next one crops up, until you are at full utilization all the time. This is not a new idea to anyone with a mainframe background. I once knew a mainframe capacity planner that would predict things like “On March 29th of next year, we’ll hit 100% average workload all during prime shift”, and be eerily accurate.


[subtitle3] When? [/subtitle3]

That all of this will come to pass I have little doubt. The issue is when. I think about such things now, because as I design new data centers, and read about DC technology, it is getting more and more esoteric. When will hot / cold aisle be out of the game, and liquid cooled racks be required? Long enough from now that I can avoid that under-floor infrastructure cost now?, or just assume I can move to something more modern every time I hit the next bottleneck? What does that mean about how I might capitalize a DC build out? Is it still a 10 year investment? We used to do these for 30 years. No way we’d do that now.



This article was written by Steve Carl, Sr. Manager, Global Data Centers, Server Provisioning at BMC Software carl-scott of BMC

See Carl’s other blog articles

“Adventures in Linux” :
“Green IT” :