Testing by turk
Posted by Matthew Evans on 21 February 2008
Testing our web applications has always been a difficult area for us at Solid State Group. When we were smaller we used to pass on the QA stage of the web build process to developers that were working on different projects in order to get fresh eyes on the site. Then came the time to hire dedicated testing resource for QA testing and to give good feedback on developer unit testing. Dedicated testing resource makes a difference to the quality of the end product and we are always keen to fnd new efficiencies in this area.
We've been usng selenium for some time now for post build testing and regression testing after adding new functionality to a project. Selenium is a great tool for automating tests that you know should return certain conditions after certain triggers, but it's never been very good at initial release testing. Trying to build the tests for selenium always raised more bugs than actually running the tests.
It seems that there is no replacement for the human eye and the ability to understand the complexities of an application, however with humans comes the overhead of management and transfer of domain knowledge.
Wouldn't it be great if the interface into testing were automated and mechanical but the actual execution were done by humans? Like the mechanical turk for testing! Well theres a new testing sheriff in town and he's called utest. This new service, to be launched in BETA in April does exactly that. It allows web applications to be tested by a community of testers regitered on the utest site. Projects are specified technically and uploaded to the site, and registered testing crews log bugs against your application. Once you verify they are in fact real bugs and not just text pasted from slashdot then the tester gets paid for that bug.
Personally I think it's one of the better uses of the so-called "social networking" revolution and we are applying to be in the BETA. It's going to be very interesting to see how the test scripts are defined and how the interaction between testers and testees works out.
Let's hope it's not just a load of the latter.