The ultimate intuitively non-obvious decision is whether to make or buy your computer service. Complicating the decision is that the decision isn't either-or but rather a spectrum of choices.
For example, suppose you want an apple pie. You can go to the bakery and buy a freshly baked apple pie. You can also go to the grocery store and buy a frozen pie, which you can either zap in the microwave or bake in the oven. You can also go to the store and buy the flour, sugar, apples, bring them home and bake a pie yourself. Finally, you can grow the apples, wheat, and sugar cane, grind the wheat into flour, refine the sugarcane into sugar, and then and only then bake the pie yourself. These are all examples of the make/buy decision for apple pies. When my wife lived in Wenatchee Washington, she bought dead-ripe apples directly from the farmer that grew them, wheat flour from a local cooperative, and only the sugar from the grocery store. Now that she lives in Seattle, she buys all the ingredients and bakes it herself. But when I want a pie, I go get a frozen one, cook it in the Microwave, and the two of us enjoy it together. The example of the apple pie is illustrative of people in the same organization making different make/buy decisions at different times and different places.
So it is with computer systems and services. You can, if you wish, buy all of the computer systems, services, and consulting that you need. This is called outsourcing. You can also buy the motherboards, CPUs, RAM, power supplies, cabinets, fans, cables, etc. and build the computers yourself, stack them, rack them, doing your power wiring, HVAC, security, etc. if you want to. What makes sense for you is probably in the middle.
I spoke with several IT managers about this question, and I've come to the conclusion that the decision is based on what the manager is comfortable with. I watched one manager in Seattle contract with a data center firm in Texas to run a couple dozen servers for him. If a computer breaks, then people in Texas swap the servers and arrange for the computer vendor to service it. The software installation and day-to-day operation is handled in Seattle. In order for this to work well, the computers have to be standardized so that they are as identical as possible so that relatively untrained (low cost) people can do the onsite work. Microsoft and Google are building major datacenters in eastern Washington and Oregon for the same reasons: electricity and land are cheap and inexpensive help is available. These datacenters are very expensive to build (but less than building them in Redmond or Silicon Valley) but cheap to operate. These work because the managers are comfortable that the data center can be hundreds or thousands of miles away from the highly skilled sysadmins.
The people you outsource to may be more competent than you are, or may be setup to do something you aren't really setup to do. For example, buying computers from a local store that specializes in servers may be cheaper than going with a big national vendor. They can build a server to your specification and service it locally. A big national vendor may want you to ship the server to a central maintenance facility. I know one vendor services equipment onsite – however on more than one occasion the technician showed up with wrong parts or bad parts. On the other hand, the big national vendor may be cheaper because they outsource their final assembly to someplace where labor is cheap.
It makes sense to outsource work if you don't have the expertise inhouse to do what you want. On the other hand, you might have the staff already to take on the responsibility. Remember what I wrote about apple pies? You can also change your mind, if you, for example, hire the staff you need or layoff the staff you no longer need.
By way of contrast, you might not be comfortable moving your datacenter. If you outsource your datacenter operation to another company, then you lose control over the quality of the people who work there. Sure, the datacenter company may say that they've done background checks but that doesn't mean that they actually did. I worked as a contract sysadmin for one company, root access to everything in the datacenter (including the credit card processing machines), the works. The company thought that the agency did a background check on me, the agency thought that the company checked me out. This is a firm with a billion dollars in market capitalization. If you outsource, then you lose control of the priorities that the datacenter crew work to. Another customer of the datacenter may have a bigger problem than your problem and your troubles may languish.
Your datacenter may be a showcase for visitors, customers and potential customers. One company built a million dollar datacenter to demonstrate how their hardware worked in demanding real-world environments. To work, the datacenter had to be close to the sales people. The datacenter was also invaluable for developers because they could perform interoperability checks when the salespeople weren't using it. So sometimes you just can't outsource the datacenter.
There are several different virtual servers around. This book focused on open source software on commodity hardware. However, if you want the ultimate in reliability, then get an F5 BigIP Local Traffic Monitor, which includes a very nice WebUI and a hardware failover mechanism. (Author's note: In the interest of full disclosure, I work for F5 networks. However, the thesis of this book, that you can make a reliable network out of commodity parts, is the antithesis of what F5 networks is trying to do). Foundry ServerIron, Cisco LocalDirector or other content-aware switch, Cisco Arrowpoint content switched, Extreme Networks' load-balancers are proprietary virtual servers.
One of the reasons why I advocate open source software on commodity hardware is because it it cheap to purchase, although relatively expensive to setup. However, the open source solution is highly scalable: once you have it working, it is easy to copy the configuration files to multiple machines. It may be that the value to the organization such that it makes sense to go with the higher cost solution. Intangible considerations, such as reliability (who publishes reliability data that you can trust?), staffing competency, managerial comfort levels, and organizational profit goals all factor into the decision.
Ultimately, an IT manager will make a judgment call about what to make and what to buy. This book advocates a “make it yourself” philosophy, because most IT managers want the comfort that doing things oneself provides. You set the priorities, the methodologies used, the processes and procedures for getting things done. It is a lot of work. But the terrifying thing is not knowing how to do it. Hopefully, this book will give you the confidence to proceed and be the master of your own destiny.