Software Liability

The recent furore about the quality of Chinese made toys has brought an interesting subject to light: product liability. In the most recent case, Mattel, the world’s largest toy maker has recalled entire product lines because of two separate issues. The biggest issue (happened twice in two weeks), was the case of the use of lead based paint. As the Wikipedia article on lead explains, long term exposure to lead could lead to damage to the nervous system and other problems; and thus the use of lead based paint has been discontinued in most industrial nations. Thus this issue of product liability has arisen out of the manufacturing process. The second issue, was the recall of certain magnetic toys, which are dangerous if swallowed, and despite certain news reports, it is not really the fault of the Chinese manufacturers. Rather, it is a design problem – the toy designers designed a faulty product, and these toys are being recalled so that they can be fixed.

This is my round about way to get to the topic – software liability. Unlike every other product in the world, software development houses (the vendors) are effectively immune from product liability. One of the biggest factors behind security issues in computer systems are because products are not designed and developed (manufactured) correctly – leading to a multitude of security problems. But unlike Mattel, vendors do not have to recall software because of bad design or development. In fact, while many vendors do provide patches, there is no obligation for them to do so (well it does make business sense for them to do so, most of the time).

There have been many, most notably Bruce Schneier, who have argued that product liability must also extend to software, and it is the only way to get more reliable, secure software. His basic argument is, at the moment, there is currently no incentive for a vendor to make secure software; and instead, it is the end user who is forced to spend extra money in an attempt to make his computer more secure through the use of firewalls, anti virus software etc. Not surprisingly, majority of vendors are opposed to software liability. They contend that software is too complex, and that there will always be bugs. Furthermore, it is not necessarily just the software that is at fault, but the combination of software applications that are used that is at fault.

In the recent report (pdf) by the Science and Technology Committee of the UK’s House of Lords (really good read), Prof. Mark Handley from University College London, sums it up very well:

“If your PC, for example, gets compromised at the moment there is no real liability for the software vendors or the person who sold them the PC or anything else. The question then is: did the person who sold you that software or the person who wrote that software or whatever actually do the best job industry knows how to do in writing that software? If they did then I really do not think they should be liable, but if they did not then I think some liability ought to be there.”

And that is exactly how product liability in other disciplines work: if a bridge falls down, it is only the fault of the construction company if they did not follow accepted standard practices like taking shortcuts or building the bridge using poor quality materials. Likewise, it is the designer's fault if they build a bridge in an area known for earthquakes without considering earthquakes in their design. And it is the same reason, Mattel is recalling some magnetic toys – because the designers did not consider what would happen if children swallowed those magnets.

Software should be the same. There needs to be some degree of accountability. During the design phase considerations such as security , reliability, stability must be taken into account. And there are tools out there to conduct rigorous testing of software design: for example (citing a tool that I know very well), Petri nets can be used to prove whether a process is bounded or not – and unbounded processes provide a good indication that the process could experience buffer overflows or similar issues during implementation.

Similarly, development also needs to have some degree of accountability. Buffer overflows caused because there are no checks on whether the input is of a correct size or not is not usually a fault of the designer. It is the fault of the programmers who did not bother to check for it, and the QA people because they forgot to test for it. Yes, programming is still a human process, and unlike robotic assembly lines, cannot be relied upon to provide perfect results, all the time. But there should be a reasonable grounding of all programmers to deliver a certain level of quality. It’s the least that should be expected.

Off course, this does not mean that patches will not be required. But, hopefully, patches will be used to fix vulnerabilities and bugs that are beyond the basic assurances. And this would still mean that users have to take care and maintain their computers and software – just like every other product. And just like every other product, the vendors should inform the users of the correct way to use and maintain software. The issue of a “computer driving license” has often been discussed … maybe it is high time, that it is actually discussed seriously.

There is off course the case of open source. In open source software, there is often no one to sue (for liability). But I think Bruce Schneier provides the perfect middle ground: open source software that is freely distributed, installed and maintained by the user (through help from online communities) should not offer any liability protection; after all the software cost nothing to begin with. However, vendors that package and support open source software (such as Red Hat), should be liable. In the end it is about assurance: from a vendor like Red Hat you are getting assurance that a specific set of open source products that is secure and stable.

I think it is inevitable that software liability will happen; it is just a matter of when. In their recommendations, the Science and Technology Committee of the UK’s House of Lords state:

“We therefore recommend that the Government explore, at European level, the introduction of the principle of vendor liability within the IT industry. In the short term we recommend that such liability should be imposed on vendors (that is, software and hardware manufacturers), notwithstanding end user licensing agreements, circumstances where negligence can be demonstrated. In the longer term, as the industry matures, a comprehensive framework of vendor liability and consumer protection should be introduced.”