Why is waterfall still practiced within software development?

[Matt Kidd]

I know the current book promotion is about learning agile but if you are in an environment reluctant to change from it's current waterfall/whatever practices aren't you mostly just reading to educate yourself?

I ask this because I was recently hired to be the first of two developers in an internal software team. I had every intention of instituting some form of agile but the current Director of PMO is against it without providing reason.

I've been on both sides recently and I can't see why any software team does anything besides an agile methodology. From the constant updates, to being abreast of issues as they happen, to having a constantly deployable code base I am baffled as to why places exist that use waterfall.

[Junilu Lacar]

Matt Kidd wrote:I had every intention of instituting some form of agile but the current Director of PMO is against it without providing reason.

And the director's objection to Agile is stopping you because... ? At the very least, did you press the director to have a discussion about his objections? Is there anything stopping you from using agile technical practices? I would do things like continuous integration, automated testing, and other agile technical practices anyway. After all, these practices contribute to the quality of the software, regardless of whether the project is Agile or not. I find it's not hard to justify using agile technical practices even when there are objections to using agile process practices.

[Andrew Stellman]

Actually, there are a lot of good reasons to use a waterfall process, and a lot of very good teams build great software using one. Jenny and I both have at various points in our careers, and like we say in the book, we're not even ashamed of it.

There are a lot of teams and companies where going agile could do more harm than good. For example, what if you're at a company where people get blamed (and even fired!) for mistakes? Having a specification that everyone agrees to up front can protect your reputation and career, because if mistakes happen you can point to the spec and show that you're following it. That's when practices you typically see on waterfall teams (like document inspections and change control procedures) can actually help make things more efficient.

And sometimes the team members themselves are actually uncomfortable with agile, even if they don't realize it. Making decisions at the last responsible moment can feel disorganized or "loosey-goosey". Scrum teams use self-organization, where the team members decide at the last responsible moment who does what task. To someone used to being assigned a task by their boss or project manager, this can feel uncomfortable, like they're taking on way too much responsibility, and they'd rather just code and not think about planning at all. I've seen this many times when training software teams. And it may sound very strange, but a waterfall style might make more sense if the discomfort with an agile mindset and attitude is so large that the team becomes dysfunctional when trying to go agile.

Luckily, it really is possible to build good software using waterfall. Really! Here's an excerpt from chapter 2 of Learning Agile that talks about what makes effective waterfall teams work:

Waterfall really can work. In fact, if you actually know what you need up front, then writing it down first is a pretty efficient way to build software.

But it takes more than just stable requirements to run a successful waterfall project, which is why they run into so many problems. Teams that build great software using a waterfall process typically have a few common characteristics:

* Good communication, because the teams that were successful in a company that mandated waterfall were the ones that consistently talked to their users, managers, and executives throughout the project.

* Good practices, especially ones like code reviews and automated testing, which are aimed at finding bugs as early as possible in the process. They usually called this “defect prevention,” and it required teams to actively think about how those bugs got into the code in the first place.

* Drawers full of documentation that have rarely been opened, because the people on the team understand that the act of writing down the plan—and the questions that get asked during that planning—is more important than mindlessly sticking to it afterward.

And as it turns out, when waterfall projects are run effectively, it’s because their teams take to heart many of the same values, principles, and practices that agile projects follow. Projects that are run using some agile techniques and practices—but that don’t really follow the agile values and principles—often end up running into the same kinds of problems that plague waterfall projects.

This is why I keep getting back to mindset, attitude, values, and principles. Those things have a huge impact on how well an agile (or waterfall!) team builds software.]Learning

[Junilu Lacar]

Andrew Stellman wrote:

And as it turns out, when waterfall projects are run effectively, it’s because their teams take to heart many of the same values, principles, and practices that agile projects follow. Projects that are run using some agile techniques and practices—but that don’t really follow the agile values and principles—often end up running into the same kinds of problems that plague waterfall projects.

This is why I keep getting back to mindset, attitude, values, and principles. Those things have a huge impact on how well an agile (or waterfall!) team builds software.]Learning

Absolutely agree. But that makes me question the labels "waterfall" and "agile" in these cases. Is a so-called waterfall project really waterfall if the teams follow agile principles and values? Is a so-called agile project really agile if the teams don't really follow agile values and principles? Who is more righteous, the man who says "Yes" but doesn't do what he says he'll do or the man who says "No" but ends up doing the right thing anyway?

[Matt Kidd]

Junilu Lacar wrote:

Matt Kidd wrote:I had every intention of instituting some form of agile but the current Director of PMO is against it without providing reason.

And the director's objection to Agile is stopping you because... ? At the very least, did you press the director to have a discussion about his objections? Is there anything stopping you from using agile technical practices? I would do things like continuous integration, automated testing, and other agile technical practices anyway. After all, these practices contribute to the quality of the software, regardless of whether the project is Agile or not. I find it's not hard to justify using agile technical practices even when there are objections to using agile process practices.

I did press with the intention of seeing how the processes differ so I could adjust but I was met with essentially "I don't have to prove to you or tell you why I don't like agile".

Admittedly I was looking for a holistic change as that was what was proposed to me and wanted to lead thew way with many of the things you mentioned like CI and automated testing. However I felt I didn't get much support in that regard to implement even though I could have just done it. Live and learn I guess.

[Matt Kidd]

Junilu Lacar wrote:

Andrew Stellman wrote:

And as it turns out, when waterfall projects are run effectively, it’s because their teams take to heart many of the same values, principles, and practices that agile projects follow. Projects that are run using some agile techniques and practices—but that don’t really follow the agile values and principles—often end up running into the same kinds of problems that plague waterfall projects.

This is why I keep getting back to mindset, attitude, values, and principles. Those things have a huge impact on how well an agile (or waterfall!) team builds software.]Learning

Absolutely agree. But that makes me question the labels "waterfall" and "agile" in these cases. Is a so-called waterfall project really waterfall if the teams follow agile principles and values? Is a so-called agile project really agile if the teams don't really follow agile values and principles? Who is more righteous, the man who says "Yes" but doesn't do what he says he'll do or the man who says "No" but ends up doing the right thing anyway?

Completely agree!! Labels are just that and business owners at the end of the day don't care how you've organized or run your team. They just want their product completed. I'm sure examples can be found of projects following agile methodologies to a T and still failing.