How to refactor the unrefactorable

[Clifton Craig]

Hello all,

I've been miserable and having some trouble getting some refactoring done on our project here. What I want to know is where are the guides for pulling apart really tough comingled code. I've recently posted two topics about my struggle on my blog. Basically what it amounts to is this. Our project is hard to maintain because there are little if any unit tests, it was developed using legacy (Cobol) database driven practices and table structure is inherent at almost everypoint of the project. There are some good ideas but they mostly get swallowed up and undone by mounds of porrly factored code. Are there any books on refactoring impossible code or any tutorials that cover things like undoing the "rogue tile pattern", introducing dependency injection in code that uses new heavily? Talk to me...

[Paul Clapham]

Sounds like you're not exactly just starting out. But you don't say what you have seen already, so let me try pointing here:

http://www.refactoring.com/

[Clifton Craig]

Paul,

Thanx for the link. What I've seen so far is a bunch of articles describing the right way to write code, and the right way to write tests. I haven't seen any guides on what to do with the mounds of difficult code I face everyday. I've read one good book, JUnit Recipes, by J.B. Rainsberger, and that answers some of my burning questions. My main gripe (and I know it's very general) is how does one safely refactor something built from the ground up without refactoring in mind. There are a bunch of things that come up whenenver I attempt to breathe new ideas into our project including but not limited to friction from management, lack of understanding from those who are supposed to help, lack of time, and other non programming related issues. I know how to do the basic refactorings (though I am no expert and struggle in many areas). What I'm looking for is more of a survival guide to refactoring. Nowhere in the text I've read, (Martin Fowler's site, Object Mentor site, TDD articles) does it prepare me for the mess that occurs when you need to move a file into a new package and it's linked to a dozens other files which may or may not need moving with it. Nowhere have I read what to do when your project is slated for a week's time span and there's no clear way to explain to management why it'll take months before you work out what appears to be an unrelated problem in what appears to be an unrelated part of the project. Maybe I'm just venting my frustration , but it seems like there ought to be such a manual for the low developer on the totem pole that wants to refactor the beast from the ground up.

[Ernest Friedman-Hill]

One thing that's very important in the process is having good tools. Hopefully you're already using an IDE with strong refactoring support built in? With such an IDE (and experience using it) a lot of very painful, tedious work can be eliminated.

At first, it could be impossible to write true unit tests, so write application tests. Write tests that document the way the whole application works, and then run them after doing a refactoring to ensure that it still works the same way.

[Ken Blair]

Originally posted by Ernest Friedman-Hill:
One thing that's very important in the process is having good tools. Hopefully you're already using an IDE with strong refactoring support built in? With such an IDE (and experience using it) a lot of very painful, tedious work can be eliminated.

At first, it could be impossible to write true unit tests, so write application tests. Write tests that document the way the whole application works, and then run them after doing a refactoring to ensure that it still works the same way.

I couldn't agree with that more. It's not just having an IDE with strong support but knowing how to use that support as well. I've been using IntelliJ for about a month and a half now and have still only scratched the surface. There's things I know must be possible to do that would save me time but I haven't found yet. For example, replacing all usages of a certain method with calls to a different method following a certain template.

[Ernest Friedman-Hill]

Originally posted by Ken Blair:
For example, replacing all usages of a certain method with calls to a different method following a certain template.

You're looking for "Search | Replace Structurally." It rocks!

[Ken Blair]

!@#(&!(@#&(!&@#

Do you know how much time that would have saved me?

[Stan James]

I have some similar code in places. I use JUnit as a framework but wouldn't call the tests "unit tests" because we have to fire up the whole server and database and partner systems and more to run them. They are more "regression tests" at some poorly defined level of detail. But they do tell me when I break the code in refactoring. Some times.

[Clifton Craig]

I agree with you all whole heartedly. However my problems come not from the lack of good tools but the lack of good direction. What I find is it's easy to know when something's wrong but to go and make it right is the biggest challenge. Many times I feel like I don't have the perfect answer so I get hung up on that. Sometimes the right answer involves steps that are impossible to achieve in my reduced time frame. Sometimes the right answer involves tasks that shake things up for the rest of my team. For example, we use Idea for developing and Perforce for source control. (We also have a build system that is very complicated.) The other day I wanted to move a Java file to a new package and then to a new project. Something that looked on the surface to be a peripheral file move turned out to be a project because the Java file referenceda couple other Java files which were used in a few places and which also referenced a few other files. Before you know it I was two days into what should have been a two hour operation. Even more difficult was integrating my changes into source control because whenever I move a file that someone else has checked out it creates problems. (Part of the problem is we're all having a little trouble using SCM.)

Also it's easy to say put a test on the application but when it comes down to it I question where and how. It's hard to know where my testing effort would be best spent until I get knee deep into some of my projects because the specs I am given have nothing to do with use cases or user to system interactions. I am given direction from database perspecitive and specs are given in database speak. To complicate matters more, if I do something outlandish like attempt to discover or describe a use case I am over compensated with more DBMA schema descriptions and references to Word docs that explain the UI through the eyes of some table. My point is I'm working on one of those projects where everything either involves some table column or is a table column relationship and it (the project in entirety) needs refactoring from the ground up. I feel I am not knowledgeable enough to build a quality system from the ground up so in many areas I question my judgement. However I am of firm belief that any idea I come up with would be better than the table-column monster that we deal with currently. I'm wondering if anyone else has overcome a similar situation and can advise me. If I ever see the light of day on this project then maybe I'll write my own book, "How to fix a project that's totally screwed up". I may sound a little harsh on our project and it's not all bad. However there are some very deep routed problems; some have to do with programming, some with tools, and others are people problems. (I got problems myself.) Maybe I should just cool out and ride this thing to the end and see where we end up. Maybe I'm just making a stink and I don't really know what I'm talking about.

[Ernest Friedman-Hill]

Originally posted by Clifton Craig:
Maybe I'm just making a stink and I don't really know what I'm talking about.

No, there are some very real and familiar issues in what you've said. None of them have easy solutions, but all of them have solutions of one kind or another. A lot has been written recently on these topics, all falling under the general heading of "agile development." You can head over to our Bunkhouse to find reviews of a number of books that I think would help you, including "Pragmatic Project Automation" by Mike Clark and "Ship It!" by Richardson and Gwaltney. Then there's Michael Feathers' "Working with Legacy Code", which is supposed to be wonderful, although I personally haven't read it.

[Ilja Preuss]

"Working Effectively With Legacy Code" would probably be a good investment for you.

Also remember that every big refactoring is just a sequence of small refactorings. If you have trouble with a big refactoring, simply start smaller. If you can't decide on anything else, just rename some variables or methods, reduce their scope etc. You don't need to find the perfect solution now - if you just improve the design a tiny little bit every time you have to work on it, you will be amazed on how good it suddenly looks some months from now...

[Jim Yingst]

Here's a recent release that may also be worth looking into: Refactoring Databases. I haven't read it and only just heard about it, so I can't say anythign else about it really, except the reviews look good, and it seems like the subject area could well be of interest to you (Clifton).

Other than that, most of what I might say has already been said (and more) by folks above. It can be depressing looking at huge amounts of undertested and poorly-designed legacy code. But don't expect to fix it all at once - just tackle a little bit at a time. Whenever you have to change some functionality, see if it makes sense to first do a little refactoring of the code in that area. Management may object if you're perceived as spending too much time just refactoring and adding tests, without adding features / fixing bugs / whatever. But if you mix in little refactorings and add a few more test as you go, directly related to teh new features / bugfixes, there's little to object to, and the benefits will accumulate over time. Probably.