The problem is this, you have thousands of supported devices, all using shared code. The programmers that maintain the code and build new code do not have access to every supported device and rarely do I think every device is tested.
Instead the process is one of “repair when needed”, you change the inet stack so the ipw modules no longer work on ipv6? someone will report the bug and it can be fixed later. Regression and Cyclic tests can only do so much, and then those seem centered around the soft modules (usb stack, file systems etc).
Is it possible to construct a testing system that allows a quicker regression test for hardware when your a distribution like Debian or Ubuntu? The only possible people with most of the hardware is going to be the diverse community of testers themselves, and even then if the testing procedure for each new kernel version is too hard, it won’t be tested.
I think we could research and develop a system which allows alpha, beta and rc testers of ubuntu to contribute their hardware testing, allowing a visual map of hardware to be constructed with possible regressions and unsupported hardware by weighting of reported incidence. This does require a fairly strict data set when it comes to hardware devices, something more solid than even smolt.
So how would such a system work? Well there is a great deal the computer can test, without ever having to bug the user about it. A basic report about what modules loaded, what hardware is available (and how they are tied up together), then you look at what device nodes are available and if a network device doesn’t have a corresponding inet interface, then it doesn’t work. If a wifi card produces a ethernet interface, then you have a problem.
You can then think about augmenting this report with a more detailed, user interactive report. Which should include almost every class of device.
Once you’ve got your report, it’s a simple matter of sending it and collating the information for developers to mull over. There could be a webpage where you can go to see the status of hardware regressions and other useful information.
How can we possibly look at device support, when we’re not even sure what works and what doesn’t with each new version of the linux kernel?