AMAZON building the whole computing platform

[blackeagle603]

fwd'd to me. don't have the original link

--


LAS VEGAS � However big and ambitious you think Amazon’s plan to run the world’s computing may be, you should probably think bigger.


In a startling talk Thursday evening, a vice president who oversees the internal engineering of Amazon’s global computing system described how Amazon is building its own specialized computers, data storage systems, networking systems, even power substations and optical transmissions systems. In every case, he said, Amazon Web Services had developed ways to make its computers run cheaper and more efficiently than standard commercial products.


James Hamilton


“We’re changing everything everywhere,” said James Hamilton, vice president and distinguished engineer at Amazon Web Services. “All of our engineers are focused” on reducing the costs of computing. He was speaking at Amazon’s second annual gathering of A.W.S. customers, partner companies and analysts, an event that drew 9,000 people.


For Amazon, which rarely talks about Amazon Web Services, it was an unusually frank and detailed explanation of a system on which the company is thought to spend perhaps $1 billion annually (Amazon officials would not confirm this number, but did not dispute it when asked directly.)


Amazon’s data storage system, Mr. Hamilton said, handles 1.5 million requests a second, and holds trillions of “objects,” or individual stored items. Its custom-built storage machines pack their systems far more densely than the most advanced commercial systems, he said, and weigh over a ton. The first database system he’d worked on professionally, Mr. Hamilton said, was considered cutting edge when it ran 69 transactions a second.


A.W.S. has built power substations with capacity in excess of 50 megawatts to have sufficient power for the computing it expects to have within two years. Amazon has hired power engineers to rethink power redundancy in cloud computing. With luck, that may help avoid the kind of disruptions<http://www.nytimes.com/2012/07/02/techn ... ?_r=0>that have taken the system down in the past.


Mr. Hamilton later said privately that Amazon had developed original statistical methods to limit damage from catastrophic failures.


Every significant component, including semiconductors and disk drives, is purchased directly, to keep the price down and to manage a chain of supply, assembly and installation that adds every day the equivalent of all the computing Amazon owned in 2004. “Every day. On the weekend, too,” Mr. Hamilton said.


In part, the talk was an acknowledgement that large competitors likeMicrosoft<http://bits.blogs.nytimes.com/2013/10/0 ... -the-past/> and Google<http://bits.blogs.nytimes.com/2013/05/2 ... ers-bloom/> are now gunning for A.W.S. With competitors, Amazon now has to engage in a customer outreach that fronts Amazon’s technical prowess.


Big public cloud computing “is such an obviously good market that we know every major player will be in it,” he said. “We still want all the jobs on A.W.S.”


Mr. Hamilton also stressed that global clouds operated with a scale and velocity that enables efficiencies individual companies can’t get running their own cloud systems. For example, by handling the peak loads of tax preparers in April and the peak loads of retailers in December, A.W.S. servers run at a higher and more efficient rate, annualized, than any individual company can realize.


This was an indirect attack on VMware<http://bits.blogs.nytimes.com/2013/08/2 ... he-future/>, Dell<http://bits.blogs.nytimes.com/2012/10/1 ... nnovation/> and other companies offering private clouds to corporations. Scale was also the central reason for many of the technical achievements Amazon listed in the talk.


Mr. Hamilton also said A.W.S. now owned its own optical fiber systems, which should enable it to manage the system on a global basis, and possibly improve its sometimes spotty performance. “We invest in long-haul fiber,” he said. “That’s happening.”


In addition to purchasing commercial systems, A.W.S. has developed its own series of networking methods that Mr. Hamilton said “are now on a Moore’s Law path,” meaning performance relative to cost is doubling every two years, and operating on systems with 64 computing cores per chip. Currently no major commercial networking company consistently achieves that sustained cost performance.


While it’s possible that Amazon will continue its openness and share its discoveries with the world. Mr. Hamilton also said Amazon was not interested in participating in industry standards discussions, which could publicize its many technical advances.


“Standards say, ‘you must do this,’” he said. “We don’t like ‘you must.’”


In fairness, Google publishes some technical papers about its big computers, but does not talk about things like the semiconductors it has developed internally to handle computing at this scale. There are so few companies able to operate that this level that every aspect of systems, even electricity usage, is considered by these companies to be potentially significant proprietary knowledge.


The notable outrider among the giant computers is Facebook<http://bits.blogs.nytimes.com/2013/01/1 ... isruption/>, which isn’t selling its own system. Instead, Facebook is focused on pure cost-cutting, and spearheads the Open Compute Project, a kind of open-source, cloud-computing architecture. Open Compute is far enough along that companies like Hewlett-Packard, which came late to cloud computing, use aspects of it in their public clouds.

[free_me]

What *is* it about Seattle based companies?

[Windy Wilson]

What *is* it about Seattle based companies?

You know, I think this is what Bill Gates was afraid of back in the day when everyone thought Microsoft had such a lock on everything that it was a monopoly and needed to be cut down a size or two. Back when whatever the competition had actually rolled out, Gates would hold a press conference and promise "vaporware", saying "We'll have this very app in three months, and it will be bigger, better, use less memory, and pour your drinks for you (So don't even think of leaving the Windows platform)."

[Jericho941]

See also: Steam console.