When it Comes to QA, Trust the Crowd

by Brian on February 27, 2009

in In the News,Only in Israel

As head of a technology startup a few years ago, one of the most important components of our software development was quality assurance – the process of looking for bugs and reporting them to the programmers to fix.

Like most startups, we were on a pretty tight budget. Hiring a team of two to three QA staff would have cost a chunk of money we didn’t have. So I wound up doing the checking myself, on top of all my other founder duties, putting in 80-hour weeks pouring through spreadsheets and bug tracking software.

Would that uTest were around at the time.

uTest is a new Israeli startup with a unique mission: to allow both small companies and more established ones to save money by “crowdsourcing” all their QA needs.

Crowdsourcing refers to the idea that functions that were once performed by an individual or team within a company can be done more effectively – and for less expense – by outsourcing it to an undefined and generally large network of professionals via the Internet.

Using the wisdom of crowds

As the Internet has grown, applications for crowdsourcing has expanded along with it. Wikipedia is probably the best-known example: Thousands of volunteers have collaboratively created the world’s most used encyclopedia. Other sites include Threadless.com, which aims to offer more fashionable t-shirts by opening up design to its community members, and CrowdSpirit which is using the “wisdom of crowds” to develop a range of $200 or less electronic devices.

uTest may turn out to the most profitable of the bunch – at least that’s what Longworth Venture Partners and Egan-Managed Capital seem to think: they led a Series B funding round of $5 million in December last year. That’s on top of $1.7 million the company raised earlier in the year.

uTest is playing in a lucrative field. According to research firm Gartner, software testing is a $13 billion business, and bugs cost US companies some $60 billion.

The uTest service is deceptively simple. Doron Reuveni, uTest’s co-founder and CEO describes the process: A company posts a QA request to the uTest platform. uTest then disseminates the request to the 12,000 testers in its community. The company can limit the request to only certain types of testers it wants – mobile, Windows and Macintosh are some of the parameters – or it can browse testers’ resumes and select only specific individuals.

Once a group of testers has signed up, the company needing the QA pays only when a bug is found and “approved” by the company. Payment is not static. Reporting a “showstopper” bug pays more than a lower level problem. Testers also earn more based on their past success records. The minimum average payment for a bug, Reuveni says, is $6-$7 but it can quickly climb to $20 or more.

Why wouldn’t a company simply take the results of a tester, not approve it and therefore not have to pay a penny? “Companies want good results,” Reuveni says. “If they don’t approve bugs, testers won’t come back. It’s all based on the rules of a social community. We sometimes find companies paying the testers even if they reject the bug in order to keep them involved.”

A fifth the cost of a QA employee

uTest requires that companies spend a minimum of $1,000 per month (they can of course spend much more if they want). The $1,000 amount usually results in about 50 approved bugs and 25-30 participating testers. It’s a great deal, Reuveni explains. “$12,000 a year is a fifth of what a QA person in Israel would cost.”

Software companies seem to like what uTest offers. In less than a year, uTest has signed up 50 customers. Clients include 888 (the world’s largest gaming and casino company) and Babylon in Israel, Eminent in France, and Move Networks, Xobni and On24 in the US.

The 42-year-old Reuveni, who founded uTest with fellow Israeli Roy Solomon, knows what he’s talking about. The Technion-educated Reuveni managed QA teams at companies including Blue Security and Enigma before starting uTest.

uTest doesn’t have much competition. Companies can find individuals on sites like eLance and RentaCoder, but there’s nothing like uTest’s marketplace matching up virtual testing teams and the companies that need them. uTest also runs educational “webinars” and regular “bug battles” to give the testing community a fun break.

uTest can keep costs down, in part, by tapping into a community of testers in developing and less expensive countries. Jeff Howe, author of the book Crowdsourcing: Why the power of the crowd is driving the future of business, points out that more than half of all freelance testers reside outside the United States.

The economic downturn, ironically, may serve to be a boon to uTest. With layoffs and cost cutting spreading across the high-tech world, a solution that actually reduces internal overhead will be hard to resist. uTest seems poised to survive the test of these tough times.


This article originally appeared on Israel21.org.

Leave a Comment

Previous post:

Next post: