I have a hint of what I think I know why people are excited about this, and one way to look at the future of something is to find an analog from the past.
Does anybody know what this is? It might not be immediately obvious. This is an electric power generator, an old one. I was in Luxembourg recently and took a tour of a 300-year-old brewery. Their business, of course, is making beer, and about 100 years ago, they were one of the first factories in Luxembourg to start using electric power to help in the manufacturing process.
Of course, they couldn’t buy the electric power off the electric grid because there wasn’t one, so they started making their own electric power. Many companies did that in that era. If you could make your operations more efficient or do new things with electric power, the only way to get power was to set up your own generator and become an expert in electric power generation.
The important thing to notice here is that the fact that they generated their own electric power did not make their beer taste better. What startup companies want—actually, companies of all sizes— is to go from their idea or vision to a successful product as quickly as possible. The problem always is that there’s a lot of undifferentiated heavy lifting that gets in the middle between your idea, vision, and that successful product.
The undifferentiated heavy lifting, by the way, has to be done at world-class levels of excellence, or your vision will fail. But it’s totally undifferentiated and isn’t actually making the beer taste better.
In the case of data centers, which I think are a pretty obvious analog for electric power generation 100 years ago, this price of admission—the kinds of things you have to do at a world-class level—are very hard. It’s very complex to build infrastructure, build scalable infrastructure, and build sensible infrastructure. It’s always changing—operating system versions always have to be upgraded.
You end up with heterogeneous environments inside your data centers, which further complicates things. As you scale, you must start forecasting for capacity planning: How many new servers am I going to need three months from now? How many new servers am I going to need six months from now? In that world, things get very complex.
And of course, you don’t get to go through this undifferentiated heavy lifting just once. The successful companies are the ones who iterate as quickly as possible. They deploy their vision in the form of a product, and then they have to change it, iterate, and with each one of those iterations, there often are infrastructural pieces that have to be upgraded, changed, or scaled. This can really slow down the pace of invention and innovation.
I’ll give you a few examples of companies that are using these services today and are basically driving this innovation loop—this success loop—faster because of these services.
The New York Times took their entire archive of articles, going back to 1851, which was four terabytes of data. They had it in a format that wasn’t easily usable. They wanted to convert all four terabytes into PDFs as a first step. It was quite a computationally intensive process with so much data—11 million different articles.
It wouldn’t have made any sense for The New York Times to set up a fleet of servers to do this task because it was a one-time task. Once the task was over, they would want to undeploy that fleet of servers—something very easy to do with the Elastic Compute Cloud.
Once they had converted all 11 million articles to PDFs, they made them accessible through this archive website using Amazon S3. All the articles are stored in S3, and S3 serves these articles. You can do a search—there’s an interesting search you can try on The New York Times archive website.
Search for “computer,” and you’ll find this: a May 2nd, 1892 reference to a “computer.” As most of you in the audience probably know, in that time frame, a “computer” meant something quite different from what it means today.
“A civil service examination will be held May 18th in Washington and, if necessary, in other cities to secure eligibles for the position of ‘computer’ in the National Almanac Office, where two vacancies exist—one at $1,000 and the other at $400. The examination will include the subjects of algebra, geometry, trigonometry, and astronomy.”