Back to article
Amazon.com To Don Red Hat Data Center
The Full Monty
August 21, 2002
Bookseller behemoth Amazon.com is getting set to put its monstrous database on Linux instead of HP-UX. In doing the heavy lifting, Amazon.com's IT staffers will work alongside Red Hat, just as they did during an earlier migration of Amazon.com's entire application and middle tiers.
"The future is pretty clear. Why not go for the 'full Monty' in the data center?" asked Michael Tiemann, chief technical officer for Red Hat. In just about the same breath, Tiemann mentioned "Oracle on Red Hat."
For the earlier deployment, Amazon.com ultimately picked Linux as the most cost-effective way to ramp up its online operations. Internal consensus didn't come easily at Amazon.com, though, back in the year 2000.
"The challenges were really mind-boggling. Some people said, 'Maybe it's impossible,'" noted Jacob Levanon, Ph.D., Amazon.com's director of systems engineering for REDIT.
Before Amazon.com took much action, Red Hat did a standard cost-benefit analysis, finding that Linux migration could boost Amazon's price performance by 4X, or "twice the performance at half the cost." Since then, some customers' estimates are ranging even higher, from 6X even up to 20X or 40X, depending on the application, Tiemann maintained.
Consequently, Amazon.com didn't find it all that tough to come up with a business rationale for the initial Red Hat migration, according to Levanon. "Why spend billions, if we can spend (only) millions?" he quipped.
The technology, though, got mired in controversy. "You have to understand that Amazon is a vast deployment," Levanon said. Initially, Levanon and some other Linux advocates at Amazon wanted to "migrate the hell out of it--migrate everything," including the data center, as well as applications and the middle tier.
"We had a lot of skeptics within the company, however, who thought we were going down the wrong path," he said. Initially, the cynics included six Microsoft administrators. By now, though, two of those six have turned more toward Linux, and another two have actually changed into avid Linux buffs, according to Levanon.
The looming holidays brought big complications. A major reason for seeking more scalability was the need to handle peaks in traffic during busy periods like December.
Amazon.com compromised in the end, opting to port "90 percent of the implementation"--everything but the data center--over a 120-day period that began on September 1, 2000. Amazon.com used Red Hat's Kickstart architecture to finish most of the work prior to the pre-holiday season.
"We had to install hundreds of servers, and migrate all our apps," without disrupting "millions of active customers," Levanon continued. Before existing apps were ported, debugging was needed. "No wonder (some people) were very leery."
Back then, more ISVs were hesitant about Linux, too. "Amazon was very instrumental in convincing ISVs," Tiemann noted. "If you want to know why so many ISVs have signed on (with Red Hat), Amazon is a company you can thank for that."
Amazon.com and Red Hat set up an implementation plan, and then followed it to the letter, according to Levanon "A good plan and execution is what it takes." Migrators from the two companies spent "days in a row without sleep."
Yet all throughout, Amazon.com kept the Red Hat consultants away from actual customer data, owing to privacy concerns. "We could not get our hands on a machine," Tiemann recalled. "That's the reality of enterprises. 'You have to play by our rules.'"
The resulting Linux implementation is more stable and easier to use. It also generates fewer trouble tickets, according to Levanon.
The deployment "significantly reduced our expenditures on technology," added Levanon, who expects costs to drop further in the future, since Linux is "a long-term proposition." Meanwhile, Amazon.com is reaching "record revenues," he added.
In a regulatory filing during the third quarter of 2001, Amazon.com reported that its technology and telecom expenditures had dropped by $17 million - or 24 percent from the previous year - due in part to the move to Linux.
In a recent report from analyst group IDC, Joe Barker, senior systems engineer at Amazon.com, is quoted as saying that Linux made it possible to replace $60,000 Unix boxes with $10,000 PC servers.
"Our goal, now, is to move to Linux on the back end," Levanon remarked.
Tiemann suggested that Red Hat is equally enthusiastic about a data center implementation at Amazon.com, partly because of the added hands-on experience this will bring.
"Technologists are building failover (and) clusters," according to Tiemann. Red Hat will work with Amazon.com "however deeply they want to draw Linux into the data center."
The database deployment, though, will "require a roadmap which can handle the increased ISV requirements," Tiemann pointed out.