Technical debt and causes of software instability

“Technical debt” refers to releasing software products that have significant bugs in them.  The cost of removing the bugs after release must be paid if the software is to survive in the long run.  So the concept is similar to financial debt: the longer you leave it unpaid, the more you must pay in compound interest.

In a recent interview, one of the authors of the term “technical debt” had this to say:

Most companies don’t have a clue on how to get rid of bugs before release. … they’re acting out of ignorance because they don’t know what they’re doing. … the companies that do know what they’re doing … don’t have much technical debt … because they’re using … defect-prevention, pretest removal, static analysis and inspections, and really good testing.

– Capers Jones http://bit.ly/WnkYap  as quoted by Joe McKendrick http://zd.net/VjuA38

Evidence of technical debt is all around us:  Maintenance efforts that are larger than the development effort on a product suite.  Failures that keep occurring in software that has been on the market for years.

Three false causes for software instability.

1.    Users don’t care

Many products are delivered quickly to an audience that is not very discriminating.  For example, if your iPhone app fails once in a while, you aren’t too disturbed unless the failure causes the whole system to stop.  And if the app allows malware to infect your phone, you probably won’t be able to tell that it was caused by shortcomings in the design of the app.

But in the long run, people will know whether the software is reliable and secure.  So they will drop out from your customer list without so much as a goodbye note.  You have a few months to get your product up to a reasonable level of reliability, but you don’t have years.

2.    It costs too much to do it right

Everyone knows it is impossible to test all combinations of inputs to a software system.  So we depend on educated guesses when testing.  And when a product is behind schedule (and what product isn’t behind schedule?), we short-change the testing in order to meet shipment date commitments.

But even though exhaustive testing is impossible, it is possible to have a well-crafted testing plan and to execute the plan in a timely way.  Yes, this does require budgeting enough resources for testing.  Doing it right saves a lot of headache and lost goodwill later on.

3.    We don’t know how.

There is no single accepted set of standards for how to create bugless software, so why try to follow the latest trends?  After all, even the biggest organizations (the FBI, airlines, banks for example) have had software projects fail to complete.

But there are management techniques, including Agile methods, that can minimize the risk of a failed project.  So there’s little excuse for ignoring the advances that have been made in software development.

Three true causes for software instability.

1.    It’s complex

As more and more products are designed with software at their core, the complexity of the software tends to get greater generation by generation.  This is good, in that the depth of the software leads to more sophisticated solutions.  But it also makes for complex systems that are increasingly difficult to stabilize.

2.    There’s a shortage of experienced and competent leaders

The demand for professional programmers and software designers is great and growing all around the world.  While many people are moving now into software development, the number who have at least a decade of real-world experience is not growing as fast, and many of those who do have not led large-scale development projects.  The gap is not being closed by traditional project managers, because most of them do not specialize in software development.

3.    We’re still experimenting with management methods

Agile development methods are formally only 12 years old.  And there are many competing methodologies out there, none of which has come to dominate the field.  While it is good that many different approaches are being tried, a manager who wants to be guided to the “best” method will receive conflicting advice, particularly from vendors who are each flogging their own piece of development-support software.

Shall we advise them to come back in 20 years, when we’re down to one or two leading approaches?  Not practical, because the development must go on.

If you’re responsible for software development or the products the software goes in, you’ll have to make some choices based on what’s known now.  It’s best to get advice on the methods and the tools, but not to depend on any one vendor to tell you how to proceed.

—————————–

My recent talk at SofTech, 10 Ways to Fail revealed many indicators of failing projects.  Download the slides & notes here.

Like this post? Share it!
    About John Levy

    John Levy works with senior managers in mid-sized organizations who are responsible for development and delivery of major software or hardware/software products. He helps them gain confidence that their projects will succeed.

    Development projects can fail in many ways. You need a guide who speaks the language of business and is knowledgeable about technology. John aligns Development with the organization's strategy so it will contribute efficiently to the success of the enterprise.

    John has been consulting for over 20 years. His book on managing high-tech teams, Get Out of the Way, was published in 2010.

    For more information, email him at johnlevyconsulting.com, or call 415 663-1818.
    And check out John's profiles on LinkedIn and Twitter!