How Relevant is the Homeland Security Grant?
Extending a Helping Hand
When the code-analysis firm Coverity announced that it, along with Stanford University and Symantec, were recipients of a Department of Homeland Security grant to improve the code of open source software projects, the expected jokes and critiques flew fast and furious.
Wags promoted ideas such as color-coded bug reports and airport-style searches for faulty code. More serious criticisms were leveled at the idea of the grant, since many in the US are still judging the DHS (in particular its subsidiary agency FEMA) harshly for its performance during and after Hurricane Katrina. On the opposite side of the perception argument, the funding seems to be free of any direct DHS management, which seemed to quell much of the criticism.
But even given a neutral stance by the US government agency, a few questions still remain. How exactly is this money going to be used, who are all the players, and what is the nature of their relationship with existing open source projects?
In open source and free software development, anyone is free to examine the code that makes up a software application. Software developers are welcome to make improvements or additions to the code. Under the General Public License, by far the most prolific free software license, any developer can take code, make changes, and even distribute that code as a new software application (a process known as forking). All the developer has to do is make those changes available back to the original software project.
However, a common misconception about free and open source software projects is that any and all changes made to the code will be accepted. As a matter of practice, this is never done. All open source software projects have a project lead or maintainer who acts as a gatekeeper for all proposed code changes, so the software will not be broken and will acquire features based on the overall notion of how the maintainer wants the code to evolve. Not all code is accepted; usually for technical reasons, though at times licensing or political issues will come into play.
The most famous example of an open source project lead is Linus Torvalds, the founding developer of the Linux kernel. Torvalds keeps a close watch on the integrity of the kernel, accepting or rejecting code changes (also known as patches) as they fit the direction he wants the kernel to go. Assisting him in this endeavor are the kernel maintainers, individual developers who manage various stable versions of the kernel: Alan Cox for the 2.2 kernel, Marcelo Tosatti for 2.4, and Andrew Morton for the current 2.6 version.
In the statements made surrounding the January 11 announcement, there seemed to be an implication that, using the combined resources of three grant recipients, code improvements would be made for "more than 40 open source projects." What was left unspoken was the question: will these open source projects accept the recommended changes that will be found? Are the changes going to be valid or necessary?
To answer this, it is necessary to examine the exact nature of the work the grant recipients will be doing. According to Ben Chelf, CTO and co-founder of Coverity, Stanford University has developed the automated code-checking procedures and will also provide manpower for the project. Coverity has developed the actual code-checking application that will perform the work. Symantec have more of an tangential role, working with the US government to improve the security of its code and endorse the use of open source software.
With these roles in mind, the process will work in this manner, Chelf explained. Money from the grant will allow Stanford to hire personnel who will run the code from various open source projects through Coverity's code-analysis tool. Once the analysis is complete, a report on any defects will be presented for the open source project to use. Chelf hopes that eventually Coverity's results will go straight to any maintainers on the project and fixes will be written and accepted by those maintainers.
But this kind of relationship does not yet exist.
"Are they using Coverity technology? No," Chelf said when asked how maintainers were currently implementing defects found in Coverity's reports. "But we hope to use the funds from the grant to build these relationships."
If Coverity can get open source project managers and maintainers to directly use the code-checking tools, ideally they will be able to analyze patches for bugs before they are applied to the main software code. This, Chelf emphasized, "is the best time to find defects."
For now, Coverity plans to publish the defect reports on a semi-private Web site so that any developer associated with a particular open source project can examine the list, determine if something actually needs to be fixed, and then create a fix and submit it to the project lead. This is currently the model used by Coverity for the defects they published for the Linux kernel.
In that project, Coverity published an unsolicited report in December 2004 indicating the number of defects found in the Linux 2.6.9 kernel to be 984, with six of them labeled as critical. In July 2005, another Coverity report found 1,008 defects--none of them critical. The results are being published on the password-protected site linuxbugs.coverity.com.
In examining recent patches made to the kernel, using patch announcements found on the Linux kernel mailing list, the number of patches based on Coverity-located bugs does not nearly approach the near-1,000 bugs found. This disparity seems to discount Coverity as a code-analysis tool for Linux.
Not quite, according to Torvalds. When asked if Coverity's results were being used, he confirmed that some bug fixes had indeed found their way into Linux patches. More importantly, the issue is not of quantity, but of quality.
"Well, Coverity doesn't send patches, but yes, some people go through the Coverity results and send patches based on them. Sometimes the warnings are bogus--automated checking has very definite limits--but just as a statistic, 'coverity' is mentioned 75 times in the commit messages over the last eight months or so," Torvalds stated, adding: "Not all of them were bugs--some of it was to just shut up a bogus warning, but the point being that it does actually end up being useful."
As a passive source of defect information, Coverity is getting some use in the Linux kernel development process. And it is not the only automated tool being used.
"We have other tools, too, for automated code cleanliness," Torvalds indicated. "Just for fun, I checked how often 'sparse'--one of the other tools--was mentioned, and it got 223 hits, and almost all of those were cleanups rather than any actual bug fix."
Coverity definitely wants to be more than a passive source of information. Chelf wants his company's product to play a more active role in the development processes of Linux and other open source projects. To do that, he hopes Coverity and Stanford can establish direct relationships with open source project managers.
"This is really a great way to reach out to a new community that develops software in a completely different way than commercial software," Chelf said.
Given the non-commercial aspect of many open source projects, however, how does Coverity plan to generate revenue from assisting open source software, beyond the initial government grant? To accomplish this, Coverity seems to be aiming for becoming ubiquitous for code-analysis. While assisting open source is a beneficial thing on its own, Chelf explained, he also recognizes that many open source developers also work with commercial software makers. If Coverity can achieve heavy use amongst the overall developer community as a familiar tool, then the company will be able to generate revenue through commercial software vendors.
In the meantime, Chelf hopes that Coverity's offer to help will be seen as just that, and not as a way to generate negative press for Linux and open source projects.
�We are here to help," he said. �We are not here to stir up the pot and make [open source developers] feel bad."