Blog Posts Case Management Process Management Process Modeling

Complex Project Delay

Blog: Collaborative Planning & Social Business

I run large complex software projects. A naive understanding of complex project management can be more dangerous than not knowing anything about it. This is a recent experience.

Setting

A large important customer wanted a new capability. Actually, they thought they already had the capability, but discovered that the existing capability didn’t quite do what they needed. They were willing to wait for development, however they felt they really deserved the feature, and we agreed. “Can we have it by spring of next year?” “That seems reasonable” I said.

At that time we had about 16 months. We were finishing up a release cycle, so nothing urgent, I planned on a 12 month cycle starting in a few months. I will start the project in “Month 12” and count down to the deadline.

We have a customer account executive (lets call him AE) who has asked to be the single point of contact to this large, important customer. This makes sense because you don’t want the company making a lot of commitments on the side without at least one person to keep a list of all and make sure they all are followed through on.

Shorten lines of communication if you can. Longer lines of communication make it harder to have reliable communication, so more effort is needed.

Palpable Danger

The danger in any such project, is that you have a fixed time period, but the precise requirements are not specified. Remember that the devil is in the details. Often we throw about the terms “Easy to Use” “Friendly” “Powerful” and those can mean anything in detail. Even terms that seem very specific, like “Conformance to spec XYZ” can include considerable interpretation by the reader. All written specifications are ambiguous.

The danger is that you will get deep into the project, and it will come to light that customer expects functionality X. If X is known at the beginning, and design can incorporate it from the beginning, and it might be relatively inexpensive. But retrofitting X into a project when it is half completed can multiple that cost by ten times. The goal then is to get all the required capabilities to a suitable level of detail before you start work.

A software project is a lot like piloting a big oil tanker. You get a number of people going in different, but coordinated directions. As the software starts to take form, all the boundaries between the parts that the people are working on gradually firm up and become difficult to change. As the body of code becomes large, the cost of making small changes increases. In my experience, at about the halfway point, the entire oil tanker is steaming along in a certain direction, and it become virtually impossible to change course without drastic consequences.

With a clear agreement up front, you avoid last minute changes. The worst thing that can happen is that late in the project, the customer says “But I really expected this to run on Linux.” (Or something similar). Late discoveries like this can be the death knell. If this occurs, there are only two possibilities: ship without X and disappoint the customer, or change course to add X, and miss the deadline. Either choice is bad.

Danger lies in the unknown. If it is possible to shed light and bring in shared understanding, the risk for danger decreases.

Beginning to Build a Plan

In month 12, I put together a high level requirements document. This is simply to create an unambiguous “wish list” that encompasses the entire customer expectation. It does NOT include technical details on how they will be met. That can be a lot of work. Instead, we just want the “wishes” at this time.

If we have agreement on that, we can then flesh out the technical details in a specification for the development. This is a considerable amount of work, and it is important that this work be focused on the customer wishes.

I figured on a basic timetable like this:

Step 1: one month to agree on requirements (Month 12)
Step 2: one month to develop and agree on specification (Month 11)
Step 3: one month to make a plan and agree on schedule (Month 10)
Step 4: about 4 months of core development (Months 9-6)
Step 5: about 4 months of QA/finishing (Months 5-2)
leaving one month spare just in case we need it. (Month 1)

Of course, if the customer comes back with extensive requirements, we might have to rethink the whole schedule. Maybe this is a 2 year project. We won’t know until we get agreement on the requirements.

Then AE comes to a meeting and announces that the customer is fully expecting to get the delivery of this new capability in Month 3! Change of schedule! This cuts us down to having only 9 months to deliver. But more important, we have no agreement yet on what is to be delivered. This is the classic failure mode: agreeing to a hard schedule before the details of what is to be delivered is worked out. This point should be obvious to all.

The requirements document is 5 pages, one of those pages is the title page. It should be an afternoon’s amount of work to read, gather people, and get this basic agreement.

Month 12 comes to an end. Finally toward the middle of Month 11, the customer comes back with an extensive response. Most of what they are asking for in “requirements” are things that the product already does, so no real problem. There are few things that we can not complete on this schedule, so we need to push back. But I am worried, we are six weeks into a task that should have been completed a month earlier.

Deadlines are missed one day at a time.

We’ve Got Plenty of Time

In the end of month 11, I give a revised the requirements and gave to AE. AE’s response was not to give it to the customer. He said “let’s work and understand this first, before we give it to the customer.” This drags on for another couple of weeks, so we are now 8 weeks into the project, and we have not completed the first step originally planned for one month.

I press AE on this. We are slipping day by day, week by week. This is how projects are missed. What was originally planned for 1 month out of twelve, is now close to 2 months, which is now out of 9. We are getting squeezed!

AE says: “What is the concern? We have 8 more months to go! What does a few weeks matter out of 8 months?”

The essence of naive thinking that causes projects to fail is the idea that there is plenty of time and we can waste some.

Everything Depends

Lets count backwards on the dependency:

We want to deliver a good product that pleases the customer
This depends on using our resources wisely and getting everything done
This depends on not having any surprises late in the project about customer desires which waste developer time
This depends on having a design that meets all the expectation of the customer
This depends on having a clear understanding of what the customer wants before shape of the project starts to ossify.
This depends on having clear agreement on the customer desires before all of the above happens.

Each of these cascades, and a poor job in any step causes repercussions that get amplified as last minute changes echo through the development.

I also want to say that this particular customer is not flakey. They are careful in planning what they want, and don’t show any excessive habit of changing their direction. They are willing to wait a year for this capability. I believe they have a good understanding of what they want — this step of getting agreement on the requirements is really just a way to make sure that the development team understands what the customer wants.

Why Such a Stickler?

AE says: “You should be able to go ahead and start without the agreement on requirements. We have 8 more months, we can surely take a few more weeks or months getting this agreement.”

Step 2 is to draw up a specification and to share that with the customer. Again, we want to be transparent so that we avoid any misunderstanding that might cause problems late in the project. However, writing a spec takes effort.

Imagine that I ask someone to write a spec for features A, B, and C. Say that is two weeks of work. Then the customer asks for feature D, and that causes a complete change in A, B, and C. For example, given A, B, and C we might decide to write in Python, and that will have an effect on we way things are structured. Then the customer requires running an environment where Python is not available. That simple change would require us to start completely over. All the work on the Python design is wasted work which we have to throw out, and could cause us to lose up to a month of time on the project, causing the entire project to be late. However, if we know “D” before we start, we don’t waste that time.

Step 2 was planned to take a month, so if we steal 2 weeks from that, by being lazy about getting agreement on the requirements, we already lose half the time needed. It is not likely that we can do this step in half the time. And the two weeks might be wasted, causing us to need even more time. Delaying the completion of step 1, can cause an increase in time of step 2, ultimately cascading all the way to final delivery.

Coming to agreement on the requirements should take 10% of the time, but if not done, could have repercussions that cost far more than 10% of the time. It is important to treat those early deadlines as if the final delivery of the project depended on them.

Lack of attention to setting up the project at the front, always has an amplified effect toward the end of the project.

But What About Agile?

Agile development is about optimizing the work of the team to be optimally productive, but it is very hard to predict accurate deliveries at specific dates in the future. I can clearly say we will have great capabilities next year, and the year after. But this situation is the case that the customer has a specific expectation in a specific time frame.

Without a clear definition of what they want, the time to develop is completely unpredictable. There is a huge risk by having an agreed upon date, but no agreed upon detailed functionality.

Since the customer understands what they want, the most critical and urgent thing is to capture that desire in a document we both can agree on. The more quickly that is done, the greater the reduction in risk and danger.

Even when developing in an agile way, the better we understand things up front, the better the whole project will go. Don’t leave things in the dark just because you are developing in an agile way. It is a given that there many things that can’t be known in the course of a project, but that gives no license to purposefully ignore things that can be known.

Conclusions

Well run project act as if early deadlines are just as important as late deadlines. Attention to details is not something that just appears at the last moment. It must start early, and run through the entire project.

Most software project fail because of lack of clear agreement on what will satisfy the customer. It is always those late discoveries that cause projects to miss deadlines. A well run project requires strict attention to clarifying the goals as early as possible.

Do not ignore early deadlines. Act as if every step of a project is as important as the final delivery. Because every step is as important as the final delivery.