It is with ‘Weinberg’s Second Law’ that Rajnikant Puranik begins his interaction with Business Line: ‘If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilisation.’
Because, to Puranik, who has authored a book on software testing titled ‘The Art Of Creative Destruction’ (Shroff), it is disappointing that poor software often gets delivered and vendors get away with it, much the same way as with roads and vehicles.
“Shoddy road construction is on account of corruption and mismanagement, but the fact remains that it is so because the road contractors, the concerned bureaucrats and politicians can get away with it! And, so could the big business in the past, with their substandard vehicles,” he frets.
There is no financial liability associated with software that malfunctions, or does not function as expected, and practically all software products carry non-liability clause, rues Puranik. Arguing, therefore, that there has to be some reasonable liability not limited to just the price of the product, he says that in the absence of such a deterrent, even the normal expected testing is not done by the vendor before the product, or its new version, is released.
Excerpts from the email interview.
On the special characteristics of software, as a product.
Each software product is unique. It is unlike any other product such as car. Over the decades, the various aspects of car manufacture have been perfected to a great degree. Most of the components that go into the car assembly can be separately sourced from various component makers. Each component is separately tested and perfected. Once a prototype has been fully tested, cars of the same model coming out of the assembly line are similar. If a new company goes into car manufacture, they do not start from scratch; they do design the car, but they source most of the components.
On the other hand, if a company wishes to develop a core banking product, it is not as if they would do an overall design and then source the various modules. They start from the scratch. They rarely have the luxury of looking at business specifications, design and code of other similar products, unlike say for cars, for which abundant literature is available and one can always look at the insides of other cars, and do reverse engineering.
For each software product you intend to develop, you need to first have reasonably good and accurate business specifications, and then you design, develop and test each piece, followed by integrated testing.
It is like developing a car from the scratch, including all its components, and with a rider that you cannot look into other cars, or have access to literature on their specifications and design.
Of course, the complexity of a software product differs from product to product. While some may be as complex as a car, or much more complex, others may be as simple as a tricycle. But, whether it is a car or a tricycle, you are on your own – you do not have specifications and designs of others available for copying.
And since each component/ module has to be made/ developed, it also has to be tested. This is quite unlike, say, Maruti, which does not have to itself bother about the quality of the tube and the tyre it uses: that is the responsibility of the supplier.
On the need for deterrence.
Let us remember that the chances of error in a software product are relatively more. Finding and fixing all the errors in a non-trivial software product upfront is almost impossible; and it is too costly and time consuming, in practice.
Though financial liability for software errors cannot be on par with other products, there has to be some reasonable financial liability, which cannot be limited to just the price of the product. For, in the absence of this, very often, even the normal expected testing is not done by the vendor before the product, or its new version, is released. It is not unusual for the vendors to leave it to the end-users to actually test the software and report bugs.
Without the necessary minimum deterrence, unethical practices are likely to continue in the interest of top line and bottom line. Not until the companies realise that it would hit them financially if they throw untested software at users would they change their ways. No amount of ISO (International Organisation for Standardisation) or CMMI (Capability Maturity Model Integration) compliance, or rather, claimed compliance, would make any difference.
On certifications and compliance.
ISO- and CMMI-compliant merely means the companies are ISO and CMMI processes-compliant. It certainly does not mean they create quality products. The logic behind ISO and CMMI is that if you follow good (that is, those as stipulated by ISO/CMMI) processes, you would deliver good products.
However, the above logic does not seem to be working in practice. Some say this is because the concerned ISO- or CMMI-compliant companies are not genuinely following the recommended practices; they do so only on paper to get the certification. There are also those who claim that following those practices is impracticable – they are an impediment to the actual work.
There are yet others who question the very relevance of high-maturity level CMMI processes for software. They say that grafting quality practices from manufacturing – such as consistency and predictability of various factors like error-rate, deviations in the actual efforts from those estimated, computing things like standard deviations, and determining if the processes are under statistical controls – is something that is inappropriate for the software field.
It can be paradoxical when companies claiming to be at ‘high-maturity levels’ fail at creating quality software in time and within budget. Critics point out that the original goal of the CMMI processes was actually to serve the bureaucratic purpose of short-listing companies for tenders by the Department of Defence; and that if bureaucratic processes could indeed deliver quality, then the Governments all over the world ought to be the most efficient bodies.
On what the software purchasers should ask for.
In the case of products such as core banking solutions, e-Governance, Internet banking, and ERP, the purchaser should specifically demand from the vendor the following:
* Documentation of software test cases and test data
* Actual test data, and their expected results
* Automated test suite to execute test cases (if not fully automated, at least semi-automated)
* Facility for the purchaser to enrich tests if required, that is, add/supplement test cases and test data
* Where required, test cases and test data to support performance testing
Benefits of demanding the above are as follows:
* If a vendor can give the above, you have the comfort that they have indeed applied themselves to testing.
* You have the readymade sets of tests to conduct, without having to first understand the software fully, and then develop your own test cases for acceptance testing.
* You can examine their ‘Documentation of Software Test Cases and Test Data’ and check it against their software and its documentation to find out deficiency in the test cases. Ask them to enrich test cases if found deficient, and re-test.
* You have a ready platform to add your own test cases and execute them.
* This facility would be immensely useful for ‘regression testing’ – crosschecking correctness whenever any patches are subsequently applied.
