Is Agile Always Appropriate?

This article describes a recent experience with a software development project trying to adopt the Agile methodology without enough guidance. This methodology is then compared with the traditional Waterfall approach, the potential advantages and pitfalls of both being compared. It is assumed that readers have basic understanding of both methodologies.

Author: Hrayr Galoyan

Overview

There are various project management methodologies available, and each has its strengths and weaknesses. Is there such a thing as the best methodology? The one that everybody should adopt; one that would lead all of us in the software development industry to that elusive goal of making all projects successful? Some think it is Agile. It certainly has the luster of the latest and greatest; it’s like a new BMW, or the latest model of iPhone – how can you go wrong with it? But is it for everybody?

Recent Experience

One fine Friday morning, we had a meeting to discuss the plans of improving our CRM system. An overhaul was long overdue; the system was growing with almost no planning, and was becoming more and more difficult to maintain. To add complexity, the CRM was involved in many business processes in the organization, from time keeping, to project management, to billing. Changing one part of the system would affect the other parts, so everybody was reluctant to make any modifications for fear of breaking some other process.

We thought we needed to spend a lot of time analyzing how various parts of the system were interconnected and then plan and implement the changes incrementally. The surprise of the day was that we were supposed to use Agile in the process. We had a couple of guys who attended Agile training class, but nobody had any practical experience with the methodology. Nonetheless, the management decided it was time to take the plunge, so on we went, marching to the orders. We probably beat the previous record of how many mistakes can be made in Agile, but we survived, and even managed to make several improvements to the system successfully.

Here are some experiences that I thought were worth sharing:

• Agile was imposed by management, rather then selected by the technical team. This resulted in lack of buy-in.
• Overall design. Everybody on the team felt that an overall design was necessary before we start changing anything, just because the system was very complex. We were told that there is no such thing as overall design in Agile, so we cannot do it. We were only allowed to design and plan for the next sprint.
• Sprint duration was selected to be one week, which was not enough to make significant changes to the system.
• Agile rules were followed as closely to the book as possible. We later found that other teams that use the methodology interpret the rules relatively frivolously. The rules are just guidelines that can be adapted to individual teams’ and projects’ needs.
• The team as a group was supposed to be responsible for the success of the project; there was no leader or manager. Having grown in a socialist country, I have a deep mistrust to the idea of group responsibility. I have seen it not work, with dramatic results.

Figure 1. The Scrum Process (source Adaptive Project Management Using Scrum)

Agile vs. Waterfall

So which methodology is better, Agile or waterfall? I have more experience with waterfall than with Agile, but I can see some advantages to both approaches. Here are some points to consider.

Design

In a traditional waterfall process, all the design is done first, followed by all the development and testing, and finally release. In Agile, we thought there is no overall design at all, but later realized that it was not strictly true. In Agile, there is an overall strategy for the product or project; it is just not as detailed as the design in waterfall. The details in Agile are filled in individual sprints. This is beneficial as teams can focus on the big picture in the beginning of the project, and figure out the details at the time they start working on a particular feature. On the flip side, without figuring all the details out, it is more difficult to assess how much effort would be required, so it is more difficult to quote the project accurately to the customer.

Scope Creep

The more documentation is available at the time of getting the customer’s approval, the better the development team is protected against scope creep. In waterfall, the design is done at the beginning of the project, so it is easier to provide adequate documentation for the customer’s approval. On the other hand, what if the team produces beautiful and very detailed document, which doesn’t leave any question unanswered, and the customer doesn’t approve the project? All the efforts would be wasted. There is always a tradeoff about how much documentation can be done up front.

Involvement of Project Stakeholders

Agile requires higher involvement of all stakeholders throughout the project lifetime, compared with waterfall, because in Agile, the design and feature selection needs to be done for each sprint. This may be difficult for stakeholders in high executive positions who may have many competing priorities.

Communications Intensity

Agile requires more intensive and frequent communication within the development team as well as with all project stakeholders. This promotes team spirit, and awareness of all team members of the project status, but could be more challenging for distributed teams.

Documentation

Documentation is critical for future support and development work on the project. Just like overall design vs. mini-design for each sprint, the documentation in Agile is maintained in each sprint, but there must be an overall strategy and understanding about what the final document is expected to contain, what its structure is going to be, and what are the standards to be followed. Executed properly, Agile can produce a documentation as good as any, but it does require extra effort and discipline compared with waterfall in order to ensure consistency.

Distributed Teams

For distributed teams, especially if they are in different time zones, it may be more difficult to follow Agile methodology, simply because Agile requires more communications. Add to this the language issue, in case of outsourcing to another country, and things may get even more complicated. Distributed teams need to think long and hard before deciding to use the Agile methodology.

A Possible Compromise

Wouldn’t it be great to be able to take the best of both worlds: the predictability of waterfall and the flexibility of Agile? There may be a way. The key is in separating the design into a project of its own. It is a matter of sales and cash flow, more than development methodology or project management, but it is a good consulting practice that benefits all project stakeholders, including the client.

This approach is mostly applicable to consulting companies providing services to external clients, but can also be used for internal projects.

When a team receives a request from a potential customer, they evaluate whether the request is simple enough so that a development proposal can be provided without spending too much time on analyzing the requirements. If yes, well and good. If, on the other hand, it will take considerable time to even understand the scope of the project, a two-phase approach is suggested to the client. In a first design phase, the team provides a fixed fee proposal to analyze the requirements and prepare detailed design documentation. A development “guestimate” may be included along with the design phase proposal, but the client understands that this number may change.

Once the design phase is awarded and completed, the team prepares another proposal for the development phase. The client can accept or refuse the development proposal. The design document is theirs to keep and to use with another development team, if they so desire.

The first phase, the design, does not involve any coding, but requires intensive communications with all project stakeholders and is Agile by nature. The second phase, the development, can follow any methodology the development team is comfortable with. This approach is more suitable for distributed teams, or outsourcing setups, because the design can be done intensively on site in a relatively limited time period if necessary. The development can be more detached, with only a limited need for communication with the client. After all, everything was already clarified and documented.

Conclusion

Your development methodology needs to be selected based on specific needs of the teams and stakeholders involved, and depending on the type of projects under consideration. Waterfall may be more appropriate for fixed fee engagements with external clients, while Agile may work better for internal projects, or time and material (T&M) engagements with external clients. There is no holy grail of development methodology. The most important thing though: don’t select a methodology just because it is popular.

Further Reading

Agile Software Development on Wikipedia: http://en.wikipedia.org/wiki/Agile_software_development

Agile Manifesto: http://agilemanifesto.org

Agile Checklist: http://www.versionone.com/white_papers/checklist/

Waterfall Model on Wikipedia: http://en.wikipedia.org/wiki/Waterfall_model

Author’s Bio

Hrayr Galoyan was born in Armenia, which is small republic in Caucasus region. Got two degrees: bachelor in computer science and master in business management. At age of 25, moved to Duba, UAE for more work experience and opportunities. Helped setup outsourcing branch in Armenia for the Dubai company, and eventually moved back to Armenia to directly manage the operations. Needed more challenge, so moved to US in 2010 and has helped numerous clients to integrate and automate their business systems. LinkedIn profile: http://www.linkedin.com/pub/hrayr-galoyan/0/876/93