Metrics of success in development – Part 1

January 26, 2008

How do you find out if your development organization is functioning well? Naturally, if you are getting products out on time, consistently, and the world around you is happy with the results, you have nothing to worry about.

But what if there ARE complaints? Can you determine whether you’re hearing gripes that have little to do with you? Or whether there really is room for improvement?

I’m working on a self-assessment tool, probably containing about 10 questions, that will help you evaluate whether you are on track. The way to use the tool is to answer the questions and then count the points at the end.

Here are the first three questions in my current working draft:

1. Counting only the past 3 projects and products under your management, how many have been completed on time? For each project/product not completed on time, how much later were they completed?

[4 points] for each project completed on or before the originally scheduled completion date
[3 points] for each project completed within 120% of the originally scheduled duration
[2 points] for each project completed within 150% of the originally scheduled duration
[1 point ] for each project completed after more than 150% of the originally scheduled duration
[0 points] for each project which is never completed

2. During the time that those 3 projects or products were being developed, how much turnover did you experience among your technical staff, including first-level supervisors and managers? Turnover means departed or transferred out from project teams or management without being invited to do so by you or your managers.

[3 points] less than 5%
[2 points] between 5% and 15%
[1 points] between 15% and 30%
[0 points] over 30%

3. In those 3 projects or products, how much of the planned functionality was delivered (at the completion date you used for question 1)?

[add 3 points] for each project in which you delivered all of the planned functionality, plus additional functionality defined after the start of the project.
[add 2 points] for each project in which you delivered all of the planned functionality
[add 1 point] for each project in which you delivered at least half of the planned functionality
[add 0 points] for each project in which you delivered less than half of the planned functionality

If your total points add up to 24, you have a perfectly-performing development organization and have no need for improvement. For the rest of us, the points are probably in the following ranges:
Excellent: 18 to 24 points
Good: 12 to 17 points
Fair: 6 to 11 points
Poor: 5 points or fewer

Shipping products on time is the key result that most of your stakeholders want. Shipping the product with the features and functions that they asked for — or expect — is next in line as a measure of your success. But if you are burning out your development crew — and causing turnover as a result — you won’t be able to sustain the results. Therefore, these are the first few measures that tell you whether the organization is successful and sustainable.

Next time we’ll begin looking at other measures that can tell you whether your management practices are helping you build a development organization that is consistent and successful for the long run.

Filed Under: Painless IT Tagged With: , , , , , , ,

Constant Reinvention = Survival

January 26, 2008
Nothing lasts forever. Even the best-conceived business strategies eventually become constraints on growth.

Consider Dell. “Dell succumbed to complacency in the belief that its business model would always keep it far ahead of the pack.” But the competitors got better while Dell failed “to invest in new business lines, talent, or innovation that could provide another competitive edge.” * [see Business Week citation below]

As a leader in technology or product development, you may think that your entire job is to execute well on the development plans laid out by Marketing or a strategic planning group. But you can do more. You can help the executive staff recognize that the business has other opportunities.

Consider what the Business Week authors went on to say: “Long-term success demands constant reinvention.” This means that while you’re turning out products that meet the current set of goals for functions, price and quality, the viability of the company may depend on your pointing out where innovative products or services could come from, using the brains you already have on your staff. Reinvention means re-thinking the orthodoxies everyone has accepted as the characteristics of the company. Do you have some independent thinkers on your staff who keep coming up with off-the-wall ideas? Maybe some of those ideas are actually your ticket to survival.

“Nurture the next growth platform long before it’s needed.” This means you have to carve out the budget to support the radical ideas from the operating budget you’re supposed to use for mainstream development. If you can’t convince your executive staff to fund a skunkworks operation, then you should look at having some of your key contributors doing some off-the-record investigations. Is this risky in your company? Then maybe the company needs some shaking up.

“Most [companies] don’t [nurture the next ideas]. Distracted by the demands of their current success, they re lulled into a false sense of security.” It’s easy to focus only on the tasks that will satisfy the demands of current customers and current ways of doing business. And while it’s not easy to perform on those tasks at extraordinary levels, you can get lost in gunning for the immediate satisfactions of meeting this quarter’s goals. Can you be a VP or Director of Engineering and still make time for thinking about next year’s products and the businesses that you haven’t entered yet? Consider this: who else is better positioned to view what’s possible, who is out there needing better functions or services, and what can be combined to make a new business?

I suggest taking an advocate’s role as part of your commitment to the long term success of your company. You don’t have to be a marketer or business analyst to know what’s exciting and feasible in the next generation of products and services. Carve out a niche as a visionary and a keeper of the wild ideas that can open up new busineses for your company. Do it regularly, and you’ll be twice as valuable as the person who only meets the usual goals of Development. Besides, it may help your company survive.

———-
* “Where Dell Went Wrong” by Nanette Byrnes and Peter Burrows in Business Week, February 19, 2007, pages 62-63.

http://www.businessweek.com/magazine/content/07_08/b4022074.htm?chan=search

Filed Under: Painless IT Tagged With: , , , , , , , , ,

Loss Leader

January 26, 2008

My colleague Joel Harrison is good at encapsulating learnings from his experience. In 2006, while I was visiting him at his startup company, Abrevity, he said, “You can’t justify a new product based on a cost analysis of the first-generation product. You have to have a vision.”

Joel and I had experienced the frustration of trying to create new products at a company that was in a high-volume, low-margin business — hard disk drives. On the one hand, Joel had prototyped a product that could have been the first available Ethernet-interfaced free-standing disk storage unit. I had been involved with defining a disk drive that stores and plays back video streams without a computer attached. While our company had funded the early prototyping of these products, it did not make the investment needed to launch them as consumer or end-user products.

Joel’s explanation, as I understand it, is that the company did an analysis of the cost of the first products in each case and concluded that the product cost too much to be priced reasonably in the marketplace. Now here’s where “vision” comes in. When you’re introducing a radical new product, you have to price it not based on the initial product’s cost, but based on a combination of the needs of the market and the expected cost curve as volume increases.

Companies selling services, such as cell phone service, do this all the time. To make the service workable, they have to invest a large amount of capital in infrastructure, such as cell phone towers, switching equipment, and so on. But pricing of the phone service must chosen both to make it attractive to the consumer and, when the number of subscribers reaches a reasonable target, to make a reasonable return on the investment.

The same thing is true with new products. The barrier to radical innovation and new product introduction in companies that have been operating in a low-margin high-volume environment for years is primarily a failure of imagination. They need the vision to see that (a) there is a market to be created or captured, (b) the product they have conceived is viable, and (c) initial pricing will lead to losses during the early stages of market development. Venture capital is based on selecting and funding this sort of innovation. But old companies have trouble thinking outside the low-margin, pay-for-itself-or-die product box.

That’s what Joel was telling me. If we could have planned the new businesses beyond the first product, and had got a commitment to fund the initial losses, we could have made history in disk drive marketing.

Filed Under: Painless IT Tagged With: , , , , , , , , ,

Shifting the focus to longer term

January 26, 2008

Startup organizations are typically unsustainable and barely stable, because:

1. The pressures to develop and market a first product require taking some expedient shortcuts, such as hiring the most capable, but not necessarily the most team-oriented individuals; placing all priority on getting a workable product out the door, rather than building the product for maintainability and growth; putting in the most features rather than the best-tested features.

2. The top management habitually focuses on the race between funds running out and product delivery, rather than on internal communications, employee satisfaction (except with the potential value of their options), and leadership style. The command-and-control management style is workable for the first few years, but typically fails to inspire the organization to build itself into a self-renewing structure.

3. Having a focus on delivering a product using already-developed technology, the company does not need to invest in longer-term development of underlying technologies, or in the people who will bring in a steady stream of new technology.

The short-term focus of a startup must change soon after the deliver of the first few products. Companies that fail to incorporate longer-term thinking around their third year find themselves living from crisis to crisis. This makes the company unattractive to good managers and good technologists who don’t necessarily get their jolllies from living in a startup environment, where “startup” means short-term thinking.

What sort of changes does your organization need, now that the product has been delivered? A new CEO who actually allows the organization to function as if there were competence at all levels? A seasoned technology executive who knows what to do to make the organization attractive to innovative people? A shift in emphasis to listening to customer feedback and involving existing customers in product decisions? Addition of a Quality department that actually has the teeth to delay a product introduction?

Whatever the changes needed, don’t be surprised by the shift. Two reactions to the shift are typical:
(1) “What happened to my adrenaline rush?” — the people who need crises to keep up their energy should pursue another startup.
(2) “I didn’t know that a company could actually plan and execute with the future (beyond 1 month) in mind!” — the people who are stressed by the company’s failure to plan and execute for the long term grow into steady, reliable contributors.

Filed Under: Painless IT Tagged With: , , , , , ,

Is your software on fire?

January 26, 2008

The spectacle of Dell laptops on fire in the summer of 2006 due to Sony battery problems has prodded me to think about product failures. There is nothing so attention-getting as a fire in a conference room. Few people who see this sort of failure will forget what they have seen.

Software failures may not be so spectacular, but they can be just as memorable to the people who witness them.

Examples from large-scale software systems: if you were waiting for your baggage in the new Denver airport a few years ago, you may have waited until human intervention delivered your bags, because the bag sorting system failed. And what if you dialed 911 and the call did not go through?

Examples from embedded systems: your cell phone drops a call due to a software glitch in the phone; your hard disk loses track of its position and takes an extra several seconds to recover.

Examples from everyday use of an operating system: Windows gets confused while processing interrupts from the web browser, and the browser hangs until you reboot; Outlook misses a beat and an email doesn’t appear on the screen when you expect it to.

If the computer or phone were to catch fire when any one of these failures occurs, you can bet that the manufacturer would do a massive recall the way Dell has done. But they didn’t. Instead, they let the users keep on running with a piece of software that “catches fire” regularly. If you’re like most users, you have become accustomed to seeing these fires and dealing with them. But do you like them? Of course not.

What are the consequences? Word of mouth travels quickly, and these failures have created a large population of users who resent having to use devices and software that fail. Resentment leads users to search for a better alternative. This is good for competitors who offer a better, unfailing solution.

But the whole world of software (and digital devices that depend on software) suffers from a bad image because of these failures. From consumer devices to mission-critical industrial control systems, everyone who has to deal with modern digital devices is gun-shy about failures. And rightfully so.

Is your software on fire?

There are known methods to assure that the software-dependent devices you make will not catch fire. If you’re not certain what these methods are, or who can implement them for you, you need to find someone who can help. But before you call in a consultant, be sure that you’re willing to pay the price: It takes time and money to make software reliable, just as it does in batteries and laptops. Are you ready to buy in?

Filed Under: Painless IT Tagged With: , , , , , ,

What is Product Marketing’s role in development?

January 26, 2008

A colleague asked, “Do you believe Product Marketing could be the bridge between the Engineering and R&D organizations? It seems to me that market requirements are the other piece to incorporate there and Product Marketing could add that to R&D’s specs before working with Engineering to determine what’s feasible and on what time schedule… What do you think?”

It works at the front end to have an Advanced Development group build prototypes and conceptual specs/models, with specs or requirements added by Product Marketing before giving them to Engineering. But the problems (below) don’t come out until the crunch — when you’re waiting for the next milestone in actual product development.

The problem is that Product Marketing and Engineering (the department responsible for developing products delivered to customers) have a natural tension: they have to arbitrate between what’s feasible within the time/dollar/featureset constraints; Product Marketing should have the customer deliveries in mind, and should interpret “what the customer wants,” while Engineering is responsible for determining which (and how many) features can be delivered within the cost and time constraints.

As a product is developed, Engineering will naturally come back now and then to renegotiate features vs. schedule (and sometimes $) as they uncover problems (or opportunities) that impact schedule. Product Marketing cannot act as the arbiter for this negotiation — Engineering must participate as a fully-responsible party, determining what can be delivered when. When Product Marketing has all the power in this negotiation, you either get emasculated Engineering, which won’t take any chances because they’re being second-guessed; or you get promises that can’t be kept, because Engineering isn’t really running the development process.

Filed Under: Painless IT Tagged With: , , , , ,