• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Rob Spoor
  • Tim Cooke
  • Junilu Lacar
Sheriffs:
  • Henry Wong
  • Liutauras Vilda
  • Jeanne Boyarsky
Saloon Keepers:
  • Jesse Silverman
  • Tim Holloway
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
Bartenders:
  • Al Hobbs
  • Mikalai Zaikin
  • Piet Souris

Re-Engineering Legacy Software: when to start over?

 
Ranch Hand
Posts: 462
Scala jQuery Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
At what point should you just abandon the legacy code base and start again? I have worked on projects where the desire to re-engineer is there (probably because management think it will be cheaper) but there is a 'don't touch in case it breaks' mentality so how would you approach that situation?
 
Saloon Keeper
Posts: 13248
291
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If there's a "don't touch or it will break" mentality, you didn't write enough regression tests. Documenting the desired behavior of an application through regression tests ensures that if you change something that breaks the desired functionality, a test will fail and you know that you have to do something else.

Typically you abandon legacy code when you learned from all the mistakes you made in it, and you are willing to make a long term investment. Starting all over is very costly, and will only yield positive results in the long run. You can't completely abandon what you have until the new system has proven that it can take on all responsibilities of the old system. This means that typically it's best to only redesign the old system a step at a time, by modularizing it and replacing individual modules.

What's incredibly important is that the people who implemented the old system, design the new system. You shouldn't design a new system "just because you don't like the existing code".

Here's what I would do to replace a crappy old application:

1) Write many regression tests.
2) Identify common problems in the system and design a module that tackles that issue.
3) Implement and test the module properly.
4) Refactor problem areas in the old code so that they use the new module.
5) Make sure that all regression tests pass.
6) Repeat steps 2-6 until you're happy with the modules you have.
7) Optionally rewire the shiny new modules with new code, and get rid of the old code you don't want.

For an interesting old article that offers another view on this, click here: http://www.joelonsoftware.com/articles/fog0000000069.html
 
Marshal
Posts: 16591
277
Mac Android IntelliJ IDE Eclipse IDE Spring Debian Java Ubuntu Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I think there's a general consensus among thought leaders in the industry that a "complete rewrite" of an application, while certain appealing to many developers, in most cases is not a responsible or professional thing to do. The first project I was ever on here in the U.S. was a $36M (in 1999) 5-year failure of epic proportions. Ironically, or perhaps fittingly, the project's acronym was CPR, with "R" as in Rewrite.

I already have the MEAP (Manning Early Access Program) of this book, so I hope Chris doesn't mind me sharing parts of it here.

Chris Birchall wrote:
1.1 Definition of a legacy project

First of all, I want to make sure we’re on the same page concerning what a legacy project is. I tend to use a very broad definition, labeling as any existing legacy project that's difficult to maintain or extend. Note that we’re talking about a project here, not just a codebase. As developers, we tend to focus on the code, but a project encompasses many other aspects, including

  • Build tools and scripts
  • Dependencies on other systems
  • The infrastructure on which the software runs
  • Project documentation
  • Methods of communication, such as between developers, or between developers and stakeholders

  • Of course, the code itself is important, but all of these factors can contribute to the quality and maintainability of a project.


    IMO, the last bullet is a crucial consideration. However, it is also this last bullet point that is most often glossed over by the optimism and excitement—and sometimes, hubris—that the thought of throwing away something that causes untold pain and aggravation can generate on a project team. I believe the failure to address the issues related to Conway's Law is the biggest contributing factor to "Rewrite" project failures. This is because communication and project team structure is something that most technical people think very little about, confident that skills and new technologies are enough to avoid problems of the past. On the other side of the cubicle, most managers only think they know everything about creating a good project structure and management system. Call me cynical but in my experience, both are often proven wrong.

    BTW, Chris, that probably should be "labeling as legacy any existing project that's difficult to maintain and extend"
     
    Junilu Lacar
    Marshal
    Posts: 16591
    277
    Mac Android IntelliJ IDE Eclipse IDE Spring Debian Java Ubuntu Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Skimming ahead to Chapter 3, I found this:

    Chris Birchall wrote:
    3.4.1 The case against a rewrite

    Let me make it clear right from the start: I believe that a full rewrite is almost always a bad idea. But don’t just take my word as gospel—let me try to convince you. Let’s look at some detailed arguments against rewriting.

    RISK
    ...

    OVERHEAD
    ...

    ALWAYS TAKES LONGER THAN EXPECTED
    ...

    GREENFIELD DOESN'T STAY GREEN FOR LONG
    ...



    Chris does give some good reasons for considering a rewrite. He also writes:

    Only once you’ve spent a considerable amount of time and effort on refactoring, without a noticeable improvement in quality, is it time to start considering a full rewrite.


    The only problem with this is that there are not that many people who have good enough refactoring skills (1 in 10, by my estimation) to make this determination. Getting noticeable improvement in quality is highly subjective and it depends on your definition of "noticeable". By definition, refactoring only changes the internal design of software without affecting the observed behavior. In order to actually notice improvement of quality, you need appropriate metrics that give you a good sense of the direction that your refactoring efforts are taking you.

    When the same people who don't have the necessary skills to refactor legacy code successfully are also the ones deciding and doing the full rewrite, that last part about greenfield not saying green for long will become evident very quickly.
     
    Will Myers
    Ranch Hand
    Posts: 462
    Scala jQuery Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Stephan, you say "What's incredibly important is that the people who implemented the old system, design the new system" but this seems counter intuitive. Surely if you do this you will end up with the same mistakes being made? I would have thought you need some fresh eyes to see a better way to do things.
     
    Junilu Lacar
    Marshal
    Posts: 16591
    277
    Mac Android IntelliJ IDE Eclipse IDE Spring Debian Java Ubuntu Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Will Myers wrote:Stephan, you say "What's incredibly important is that the people who implemented the old system, design the new system" but this seems counter intuitive. Surely if you do this you will end up with the same mistakes being made? I would have thought you need some fresh eyes to see a better way to do things.


    What's more, the people who implemented the old system are seldom around anymore to be involved in the design of the new system. Even when there are still people around who can give you an idea of the thinking behind the old system's implementation, it still isn't enough to help you with the new system. This kind of thinking about "fresh eyes" is the kind of mindset that I mentioned earlier, where people embarking on the rewrite always seem to think they can do better. There's a fallacy in this mindset that's not unlike the one that makes people think "Well, I haven't won the lottery in five years of playing it, I'm sure I'm going to win big this time around."
     
    Bartender
    Posts: 10780
    71
    Hibernate Eclipse IDE Ubuntu
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Stephan van Hulst wrote:Typically you abandon legacy code when you learned from all the mistakes you made in it, and you are willing to make a long term investment.


    @Will: And NOT before.

    There is, fortunately, a "fallback" position - albeit a less comprehensive one - and that is to replace legacy code where you can or when it makes sense.

    The usual characteristics are something like this:
    1. It will be a method or component that you have been asked to change.
    2. It might as well have been written in Chinese (or in Runic if you are Chinese ).
    3. You spend a week going through it; and around the 5th day a lightbulb comes on and you think "oh, THAT'S what its doing", and you spend the next 2 verifying that your lightbulb moment was, in fact, correct.

    At that point (and after consulting your bosses) you can rewrite the original code in as clear and simple a fashion as you can, test it, and THEN apply your change. And don't forget to document it copiously.
    Half the problem with legacy code is that it assumes knowledge that has been lost (and you had to "re-learn"); so don't lose it again.

    Other than that: Listen to Stephen.

    HIH

    Winston
     
    Stephan van Hulst
    Saloon Keeper
    Posts: 13248
    291
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Will Myers wrote:Stephan, you say "What's incredibly important is that the people who implemented the old system, design the new system" but this seems counter intuitive. Surely if you do this you will end up with the same mistakes being made? I would have thought you need some fresh eyes to see a better way to do things.


    I would think that people who have made the mistake and have learned from it are less likely to make it again.
     
    Junilu Lacar
    Marshal
    Posts: 16591
    277
    Mac Android IntelliJ IDE Eclipse IDE Spring Debian Java Ubuntu Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Stephan van Hulst wrote:I would think that people who have made the mistake and have learned from it are less likely to make it again.


    What can be said optimistically about people as individuals seldom translates well to groups of people, at least not at the same time scale. If they did, then we shouldn't be seeing as many project failures and new legacy systems being built as the years go by. I don't see that happening though. It seems like almost every project I've ever been involved with was either a legacy project or fast becoming one. It takes a lot of effort, dedication to quality, skill, and communication to keep software entropy at bay.

    This is what Chris wrote in the summary of Chapter 3 (emphasis mine):

    Chris Birchall wrote:Improving a legacy codebase has to be a team effort, so you need to make sure your team is communicating well and working toward common goals before you start.


    This goes back to what I said about Conway's Law earlier. I just don't see very many teams that are conscious of this important aspect of software development and actually do something about it.
     
    Junilu Lacar
    Marshal
    Posts: 16591
    277
    Mac Android IntelliJ IDE Eclipse IDE Spring Debian Java Ubuntu Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    One of the most important chapters in this book is Chapter 10 "Stop writing legacy code!" The title bolsters my assertion that groups of people aren't very good at learning from the lessons of the past. The recommendations in Chapter 10 include pair programming and code reviews. How many developers do you know who do pair programming for most of their development work? What about teams that do meaningful code reviews regularly? I bet the proportions in general would still be around 1:10 or less.

    I realize the tone of my responses so far have been pretty cynical but it comes from seeing and working with all manner of legacy code for over 20 years as a software developer. At some point in time, I'm sure I was a contributor of legacy code as well. The turning point for me was around the turn of the century in 2000, when I learned about agile software development techniques. The most important thing about agile software development or any successful development paradigm that doesn't produce legacy code is a culture of quality. Chris cites the example of the Linux kernel. It's old and large, two of the criteria for legacy software. Except the Linux kernel is not legacy code. That's because of the strong culture of quality that exists within the core team that maintains and develops it. That culture exists not in small part because of the "dictatorial" style in which Linus Torvalds communicates his intentions to everybody involved with the project (paraphrasing what Chris wrote).
     
    Junilu Lacar
    Marshal
    Posts: 16591
    277
    Mac Android IntelliJ IDE Eclipse IDE Spring Debian Java Ubuntu Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Speaking of culture, Chris discusses it in 1.4 Legacy Culture with subheadings for Fear of Change and Knowledge Silos. He also discusses factors that contribute to "a paucity of communication" which includes lack of face-to-face communication and "code ego", where people have a mindset that makes them defensive about their code and averse or resistant to having their work reviewed by other people.
     
    Saloon Keeper
    Posts: 24283
    167
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Likes 1
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Speaking of legacy, consider that way back when, Fred Brooks wrote in his book "The Mythical Man-Month" about the so-called "Second System Effect".

    I've seen a few projects fail because of this. Instead of being simpler and cheaper than maintenance, you end up with a committee writing everything, including the kitchen sink and a pony into the new system. And if you're REALLY lucky, some overpriced consultant will come in with an overpriced overrated monkeys-can-do-it-in-a-day tool and destroy what little chance the project did have, leaving a heap of rubble behind whereafter you have to continue limping on under the old system. Except that in some cases, there was a more compelling reason (for example, legal compliance), and everyone ends up truly suffering. I could cite a large local company with a 2-year deadline that spent about 18 months drawing stick-figure diagrams, realized they only had 6 months left, and had to dump the stick figures in favor of simply trying to hack something out quick.

    You have a legacy system when:

    1. Nobody is sure what it all does or how it all works (well, either legacy, or bad design)
    2. Reports come out that go straight to the trash can because nobody cares about receiving them
    3. Inputs are missing but it turns out that it doesn't make a difference
    4. Documentation (including code comments) that says "Don't touch this! We don't know what it does, but if you mess with it, it will break"
    5. It's so old, people weren't doing regression tests back then (most mainframe systems I worked with)

    I worked with one system that required punched card inputs to work. But it became apparent when we upgraded OS's, that no one really knew what those cards contained. Turns out, they didn't contain anything. The behavior of the program depended solely on how many cards were in the data deck, not what was punched on them.

    Agile is a good counter to the Second System Effect. You just take up all the kitchen sinks and ponies and say, "That's for the next+++++ milestone". With any luck, you can keep them at bay forever.
     
    Consider Paul's rocket mass heater.
    reply
      Bookmark Topic Watch Topic
    • New Topic