Pitting Cloud against Cloud

From MIT's Technology Review:
New software developed to measure the performance of different cloud computing platforms could make it easier for prospective users to figure out which of these increasingly popular services is right for them.

Duke University computer scientist Xiaowei Yang and her colleague Ang Li are trying to make the cloud market more like the car market, where, as Yang says, "you can compare specifications like engine size or top speed." Yang and Li have developed a suite of benchmarking tools that make it possible to compare the performance of different cloud platforms without moving applications between them. These tools use algorithms to measure the speed of computation, and shuttle data around to test the speed at which new copies of an application are created, the speed at which data can be stored and retrieved, the speed at which it can be shuttled between applications inside the same cloud, and the responsiveness of a cloud to network requests from distant places. The researchers used the software to test the services offered by six providers: Amazon, Microsoft, Google, GoGrid, RackSpace and CloudSites. Results of those tests were combined with the providers' pricing models to allow for quick comparisons.

The researchers aren't yet willing to disclose the performance scores of specific providers, but they plan to make their tools publicly available. "We are building a website where people will be able to download the software we used and see the results of the benchmarks," says Yang. "We gathered our initial data by running the trials for a few hours, sometimes over two days. But ideally they should run every day to provide live data on the clouds' performance."

http://www.technologyreview.com/computing/25815/