Technical debt and causes of software instability

February 7, 2013

“Technical debt” refers to releasing software products that have significant bugs in them.  The cost of removing the bugs after release must be paid if the software is to survive in the long run.  So the concept is similar to financial debt: the longer you leave it unpaid, the more you must pay in compound interest.

In a recent interview, one of the authors of the term “technical debt” had this to say:

Most companies don’t have a clue on how to get rid of bugs before release. … they’re acting out of ignorance because they don’t know what they’re doing. … the companies that do know what they’re doing … don’t have much technical debt … because they’re using … defect-prevention, pretest removal, static analysis and inspections, and really good testing.

– Capers Jones  as quoted by Joe McKendrick http://zd.net/VjuA38

Evidence of technical debt is all around us:  Maintenance efforts that are larger than the development effort on a product suite.  Failures that keep occurring in software that has been on the market for years.

Three false causes for software instability.

1.    Users don’t care

Many products are delivered quickly to an audience that is not very discriminating.  For example, if your iPhone app fails once in a while, you aren’t too disturbed unless the failure causes the whole system to stop.  And if the app allows malware to infect your phone, you probably won’t be able to tell that it was caused by shortcomings in the design of the app.

But in the long run, people will know whether the software is reliable and secure.  So they will drop out from your customer list without so much as a goodbye note.  You have a few months to get your product up to a reasonable level of reliability, but you don’t have years.

2.    It costs too much to do it right

Everyone knows it is impossible to test all combinations of inputs to a software system.  So we depend on educated guesses when testing.  And when a product is behind schedule (and what product isn’t behind schedule?), we short-change the testing in order to meet shipment date commitments.

But even though exhaustive testing is impossible, it is possible to have a well-crafted testing plan and to execute the plan in a timely way.  Yes, this does require budgeting enough resources for testing.  Doing it right saves a lot of headache and lost goodwill later on.

3.    We don’t know how.

There is no single accepted set of standards for how to create bugless software, so why try to follow the latest trends?  After all, even the biggest organizations (the FBI, airlines, banks for example) have had software projects fail to complete.

But there are management techniques, including Agile methods, that can minimize the risk of a failed project.  So there’s little excuse for ignoring the advances that have been made in software development.

Three true causes for software instability.

1.    It’s complex

As more and more products are designed with software at their core, the complexity of the software tends to get greater generation by generation.  This is good, in that the depth of the software leads to more sophisticated solutions.  But it also makes for complex systems that are increasingly difficult to stabilize.

2.    There’s a shortage of experienced and competent leaders

The demand for professional programmers and software designers is great and growing all around the world.  While many people are moving now into software development, the number who have at least a decade of real-world experience is not growing as fast, and many of those who do have not led large-scale development projects.  The gap is not being closed by traditional project managers, because most of them do not specialize in software development.

3.    We’re still experimenting with management methods

Agile development methods are formally only 12 years old.  And there are many competing methodologies out there, none of which has come to dominate the field.  While it is good that many different approaches are being tried, a manager who wants to be guided to the “best” method will receive conflicting advice, particularly from vendors who are each flogging their own piece of development-support software.

Shall we advise them to come back in 20 years, when we’re down to one or two leading approaches?  Not practical, because the development must go on.

If you’re responsible for software development or the products the software goes in, you’ll have to make some choices based on what’s known now.  It’s best to get advice on the methods and the tools, but not to depend on any one vendor to tell you how to proceed.

—————————–

My recent talk at SofTech, 10 Ways to Fail revealed many indicators of failing projects.  Download the slides & notes here.

Filed Under: Agile Management, Painless IT Tagged With: , , , , , , , , ,

No silver bullet

December 10, 2012

Software is in everything and we and our businesses depend on it more and more.  Yet Software Quality is not rising, so we have rising numbers of failure incidents and out-of-control costs in maintaining software.  What should you do about it?

Software, software, software

No matter where you look, there is software.  Whether you inspect the thermostat in your home, look at the smartphone in your pocket, or lift the hood of your car, you find digital chips running software that keeps the device going.

And this doesn’t even begin to describe all the software that is running in your computer and in The Cloud.  Software is everywhere and we are dependent on it for so many things in our daily lives.

If you have something to do with creating software, you’re probably in a secure job because software creation is not going away.  On the other hand, you’re probably worried about keeping up with the latest techniques and standards, because software development is in the public spotlight more and more.

Why?  Because software failures, system data breaches and rising maintenance costs are in the news more than ever.

Software can be stable and reliable

I attended this month’s meeting of an organization called SofTech and enjoyed hearing Fred Davis talk about the latest gadgets – which, of course, are full of software.  And in that room were some of the most experienced software developers in the San Francisco Bay Area.  Yet even among those high-tech gurus there is an unspoken acknowledgement that software quality is not very high overall, and that creating stable and reliable software is an arduous undertaking.

How can we make it less arduous?  Well, as Fred Brooks explained, there is no silver bullet — no single countermeasure that will make software development become predictable and reliable.  If you want reliable software, you have to organize and execute deliberately, monitor the results regularly and keep up with the evolving tools and methods that incrementally make the process better.

To learn more about development issues, have a look at Technical Debt.  Also visit SEI, PMI, and CISQ.  But above all, get expert guidance that is not focused solely on technology and tools, because creating reliable software depends as much on management and organization as it does on tools and process.

If you’re managing development projects …

I’ve started offering a series of webinars on managing development projects.  The first two were titled The 10 Danger Signs of a Failing IT Project and How to Fix a Failing IT Project.  The third one, in January, will be Why Agile Won’t Fix All Your Problems.

Even these webinars won’t fix all your problems.  But you may become aware of the possibilities and some of the pitfalls in development.  And that could be enough to get you on a path of improving the software quality in your enterprise.

 

Filed Under: Agile Management, Painless IT Tagged With: , , , , , , , , , , , , ,

Is your software stable or static?

July 10, 2012

In most professions, it’s good to have stability in the things you work with.  With software, stability is good, but often we confuse static with stable.  They are not the same.

Static software decays and becomes useless.

In a previous article, Why does it cost so much to maintain systems? I explained how software updates are critical to keep it alive:

Software by its nature is constantly evolving.  Not only do users ask for new and modified features, but the systems on which the software runs keep changing, so the software has to be modified to fit new environments.  These factors guarantee that updates have to be released several times a year to keep things current.

Here are some indicators that software is stable:

• Updates are regular but not urgent

• The list of bug fixes in each release is not excessively long

• You don’t experience crashes & freezes on the system

• The software “plays well with others” – there are no negative interactions with other software packages

• It scales up smoothly – there are no abrupt limits as the data size or workload increases, only a gradual slowing of response time

• The vendor’s support is consistently accessible and useful

On the other hand, static software is an entirely different thing.  If software is static it means that there are no updates, changes or evolutionary steps being taken to keep the software current.  As a result, some or all of the following will happen:

• The software will not run in new environments, such as latest release of the operating system

• The files being used with the software do not evolve with changing data formats, file systems and storage media.  Compatibility is lost with other software that begin using new data and file formats.  Old backup copies of data may not be readable on new storage devices.

• The software will not communicate using new network interfaces.  When it does communicate, it may be using old protocols that slow down the speed of transfers and do not use new features available in the network.

• The software may not be able to interact with cloud-based applications and databases that have been added since the software was released.

Stable software requires constant attention by the people who are maintaining it.  They should be in regular dialogue with the users of the software, have a list of issues they are investigating and correcting, and be using a set of maintenance tools that allow consistent and stable releases.

Strive to have stable, but not static software. 

Filed Under: Agile Management, Painless IT Tagged With: , , , , , , , , , , ,

Why is software maintenance so expensive?

June 12, 2012

When we purchase a piece of software – even as a service – we tend to think that the major expense is finished with the purchase price.  But it’s not true.  Whether you buy, build your own, or rent software, it always costs more in ongoing expenses than the initial price.  Why?

Developing software is an expensive proposition.  Not only do the people cost a lot, but they use expensive tools, and they always seem to need more time than was estimated at the front end of the project.  So we should not be surprised that software sells for a lot of money, even when the vendor is selling a lot of copies.

But why does maintaining that software cost so much?  Here are some of the reasons:

Software by its nature is constantly evolving.  Not only do users ask for new and modified features, but the systems on which the software runs keep changing, so the software has to be modified to fit new environments.  These factors guarantee that updates have to be released several times a year to keep things current.

Just because you bought the software doesn’t mean you – or the rest of your staff – know how to use it well.  So you need people to study up on the software’s functions and then teach others how to make the best use of it.  If you don’t, you won’t realize the full benefit that the software offers.

The worst thing that can happen to a software user is to learn that the vendor has gone out of business.  So you want the vendor to do what it takes to keep making those updates and upgrades available.  This means that the vendor also has to keep a stable of specialists assigned to the software product, even if it is no longer the mainstay of the vendor’s business.  And those specialists have to aware of how customers are using the product, and what kind of support they need.  So you’re willing to pay for that support, if the product is of great value to you.

When things go wrong, you need someone to call.  If you’re a big enterprise and the software is integral to your operations, you’ll want that someone to be nearby – and on your own payroll.  Otherwise, the person will not feel the urgency that you do when there’s a failure.  So you, too, have to maintain a staff of specialists who keep up to date with the software’s features and failings, and know how to work around any problem that comes up.

Furthermore, when something fails, you typically don’t know whether it was caused by a software bug in the application, a system failure or something gone haywire in the infrastructure (such as the network).  So you need IT specialists who can diagnose failures and then pursue the solutions no matter where the failure originated.

Can you avoid all of these expenses by using software in the cloud?  Not so fast.  You may save money by renting software that runs in the cloud, but you have to be ready to accept a standardized version of the software that everyone else is running, too.  Or you’ll have to pay extra for customization, which leads you right back into the ongoing costs of maintenance.

You’ll also have to endure the changes that come when the cloud-based software vendor decides to upgrade the system – on the vendor’s timetable, not yours.  So maybe you need those specialists in IT after all?  And the cost savings of the cloud are not completely one-sided.

Filed Under: Painless IT Tagged With: , , , , , , , ,