DoctorMo's Blog

I was reading an article about how Ubuntu is a bad standards barer for the “Linux” desktop. I’ll leave aside how paradoxical the brand “Linux” is used to mean desktop when it means nothing of the sort and I assume she means FreeDesktop (FDO).

Now I know that she’s biased and makes a few mistakes, for instance inferring that fedora is made by Red Hat alone, claiming that Red Hat has any sort of desktop strategy at all other than “for geeks”. But don’t let her red hattidness mean that there isn’t a good point about testing.

But I was struck by the problems that she has had and the comments to the entry. When comparing them to my own support roster for the past few months of sudden grub mortality (8 cases) where grub just looses all ability to boot anything with cryptic errors such as “Invalid symbol ‘u’ found” and ‘Error 15′.

Other problems concerning hardware or software failures are popping up more too. It’s actually quite worrying the amount of hand holding I’m having to do for new installs of Ubuntu. Not just the usual restricted drivers, restricted extras, libdvdcss and sun-java installation, but more messing about to get systems stable.

So, what do I think we should do about some of these problems? Testing is one way of getting stuff working, but our testing procedure for releases isn’t organised properly. We simply release a candidate and let users use it until it breaks, very unstructured. As a general testing that’s fine, but I think we should be asking users to sign up specifically to test specific things. Hardware they have, software they use, and that this process could well be made much easier by having more automated testing that executes greater depth of features and mimics facets of use better. If that’s not possible then having better defined instructions of how to test your hardware and having ways of inputting that into reports that are just as easy to fill out.

Lists of hardware and software, who has tested them, what the results are, what isn’t being tested (which requires us to know of every piece of hardware too). We need users who test to feel important in what they test and the reports can help us pick out issues, cut down on the bureaucracy of bug handlers between the users and the developers that simply invalidate bugs because there is a new version without even testing to see if it’s fixed. We can see what is being tested a hundred times and what never gets tested, we need to furnish developers with a way to collect together a list of testers to form an informal community to test new versions of things for patch fixes.

I think we’re on the way to making beta/alpha releases easy to install as secondary systems and packages already allow us to move between versions of a package. More work is probably needed to make it all fluid and easier. It would be nice if packages in ubuntu had live tests that exercised their installed state, something that could be run on any chipset and reported back.

I really do think we need to do far more that what we’re currently doing, and not just more testers, but more structures for QA to grow upon. But of course this requires resources, we have a community willing to do stuff but perhaps we should have some paid QA Community people who can provide the structure and really think in depth about how to exercise the community’s enthusiasm and self interest in testing new releases.

Your thoughts?

Update: People have commented quite rightly that we have a major deficit in programming time. I attribute this to lack of economic viability, others are attributing it to lack of skill in the willing participatory community. But for now I just want to talk about making testing easier and the results more useful and directly targeted at developers.