XP and controlling "scope creep"

[Junilu Lacar]

Trying to revive the discussion that was accidentally deleted.
The initial statement was something to the effect that in XP, scope creep is controlled by having the developer negotiate with the client such that if the client wants X to be included in an iteration, he must also sacrifice a Y with similar costs. (Is that about right, Ilja?)
[ June 07, 2002: Message edited by: Junilu Lacar ]

[Corey McGlone]

Originally posted by Junilu Lacar:
Trying to revive the discussion that was accidentally deleted.
The initial statement was something to the effect that in XP, scope creep is controlled by having the developer negotiate with the client such that if the client wants X to be included in an iteration, he must also sacrifice a Y with similar costs. (Is that about right, Ilja?)
[ June 07, 2002: Message edited by: Junilu Lacar ]

I believe that was the general jist of the discussion. The question I had posed was something like this:
Let's say a customer wants a given feature in the system. This feature would require 40 hours to design and implement. Now, if the customer didn't tell you about that requirement until you had already begun development, it would then cost the customer more than 40 hours, correct? The additional cost would come from the extra refactoring required to remove some other feature and add this new feature. So, had the customer come to you with the feature in the beginning, it would have cost 40 hours. But, now that you're gotten the request so late, you'll end up having to remove a 50 or 60 hour feature in order to put this 40 hour feature in place, correct?
So, even though XP allows more flexibility in requirements, there still seems to be a price to pay. Is this reasonable?
Corey

[Junilu Lacar]

No, I think you've misunderstood it. The way XP works is that the features that are most valuable to the client are included in the current iteration plan. The replacement of Y with X occurs in the plan, not in the code. Work on Y is simply moved to the plan for a subsequent iteration. If work has already started on Y, I think the client will just have to find another task/s that he finds less valuable to move to another iteration.
XP is adaptive rather than speculative. You never really design for the future in XP. Detailed design only goes as far out as the current tasks and is constantly evolving through refactoring. Refactoring and unit tests help mitigate the risks of making changes in the future and not designing everything up front. This is what Kent Beck refers to as flattening the cost of change curve.

[Ilja Preuss]

Junilu, thanks for bringing this mess under controll. Things are going a little bit topsy-turvy for me today... :roll:
You summarized the way of XP very nicely.

Originally posted by Corey McGlone:
Let's say a customer wants a given feature in the system. This feature would require 40 hours to design and implement. Now, if the customer didn't tell you about that requirement until you had already begun development, it would then cost the customer more than 40 hours, correct?

Possibly, though probably not as much as you'd expect. See below
Nevertheless, XP *is* advocating getting to know all the requirements upfront you can know off at least roughly. But it is also asserting that it can add great value to a project to be able to react to a changing business environment or new inspirations.

The additional cost would come from the extra refactoring required to remove some other feature and add this new feature.

In XP, you don't do much "upfront design". Allmost all of the design for a feature (anything more than regarding the System Metaphor) is done in the iteration it is implemented in (in fact most of it *while* it is implemented). So, removing a feature does cost you nearly nothing.
Because of the same reason, adding a feature late doesn't add much additional cost, as you wouldn't have done much differently if you knew of the feature earlier (except possibly implemented it earlier).

[Corey McGlone]

Originally posted by Junilu Lacar:
The way XP works is that the features that are most valuable to the client are included in the current iteration plan. The replacement of Y with X occurs in the plan, not in the code. Work on Y is simply moved to the plan for a subsequent iteration. If work has already started on Y, I think the client will just have to find another task/s that he finds less valuable to move to another iteration.

So, it would seem to me that you do still put a freeze on requirements, it's just not until you've started to work on the design an implementation of a specific feature that it is frozen. Is that correct?
However, if you really only design and develop one feature at a time, don't you lose some ability to take advantage of various design concepts. For example, let's say that we're developing some software for a university. In order to fulfill a requirement (let's say register for classes), we need to create a Sudent class. Now, for some later requirement, (let's say submit grades), we need a Professor class. It is quite likely that the Student and the Professor classes are going to share some characteristics so an inheritance heirarchy might be appropriate. However, it doesn't make sense to simply have Professor inherit from Student. Rather, it would make more sense to refactor those characteristics into another class and have both inherit from that. But, doesn't going through a process like this, perhaps many times, get costly? I mean, you have extra refactoring costs and you'll have to regression test the feature that utilized the Student class to begin with. It seems to me that this would increase the amount of overhead involved in such a process.
Corey

[Junilu Lacar]

If regression testing is done manually, then yes it would get costly. But in XP unit tests form the basis for your regression tests and the unit tests are automated ( see http://www.junit.org ).
WRT to design, there is still some up-front design done but very little of it. You don't spend two weeks or months doing design work then cut over to the construction phase. The detail design occurs side-by-side with construction through creation of unit tests and refactoring.
There's a rhythm to it: Write a test (up front design), see it fail (because it's not implemented yet), write code to make the test pass (construction), refactor and make it good. Repeat until satisfied.
Because you implement one little piece of functionality at a time, the process is very focused and you actually save time in the long run because you reduce, if not eliminate, the time you would have normally used to debug and test (unless of course you can write 2000 lines of code without introducing a single bug ). In each small step, if you do happen to introduce a bug, you'll know immediately what code did it: the one that you just wrote!
[ June 07, 2002: Message edited by: Junilu Lacar ]

[Corey McGlone]

Originally posted by Junilu Lacar:
unless of course you can write 2000 lines of code without introducing a single bug

What, you can't? :roll:

[Ilja Preuss]

Originally posted by Corey McGlone:
So, it would seem to me that you do still put a freeze on requirements, it's just not until you've started to work on the design an implementation of a specific feature that it is frozen. Is that correct?

Well, not exactly...
The customer *is* allowed to cancel a requirement that is currently worked on - he then just has to live with the fact that the already invested work for this feature is wasted.
It is the full right and responsibility of the customer to tell the developers which features they should work on.
It is the responsibility of the developers to elucidate the customer about the costs of his decisions and to generally hold this costs as low as possible.

For example, let's say that we're developing some software for a university. In order to fulfill a requirement (let's say register for classes), we need to create a Sudent class. Now, for some later requirement, (let's say submit grades), we need a Professor class. It is quite likely that the Student and the Professor classes are going to share some characteristics so an inheritance heirarchy might be appropriate. However, it doesn't make sense to simply have Professor inherit from Student. Rather, it would make more sense to refactor those characteristics into another class and have both inherit from that. But, doesn't going through a process like this, perhaps many times, get costly? I mean, you have extra refactoring costs and you'll have to regression test the feature that utilized the Student class to begin with. It seems to me that this would increase the amount of overhead involved in such a process.

OK, let's say you need the Student class, and because you think you will need it later, you already implement it as a hierarchy (with or without already adding the Professor class - it doesn't matter much). What could happen?
- You did it right! You actually need the Professor class in the way you implemented it. So you did save some time by doing it earlier, didn't you? Well that's not the whole picture. In fact, you did this by investing time that could instead have been used on a feature that would have brought value to the customer earlier and was thought of him as being more important! And this not only by the time used directly for the implementation, but also by making the design more complex and therefore increasing the costs of testing, refactoring and implementing intermediate stories.
- You didn't got it that right, sadly. For example you discover that a Person can be a Student and a Professor at the same time, making the Role Pattern a more appropriate choice. Not only is the time on building the inheritance tree is wasted, it even makes it more costly to refactor to the design you actually need.
- The customer finds stories that are more important than the one needing the Professor class, delaying it again and again. Perhaps it never gets implemented at all. Again, not only did you waste time by implementing the hierarchy, but also by maintaining a design that is more complex than necessary.

A True Story:
I once worked in a team of five junior developers on a three week XP training project. We had our secretary as customer and worked the first two weeks on a batch system to automatically process e-mails she so far had to evaluate by hand. We had to parse all e-mails of an account (containing ascii formulars filled out by humans, so it wasn't *that* trivial) and connect to a database to gather associated data and present the results in an spread-sheet. It was working out quiet well. For the third week we had planned to develop a nice little GUI, so that the secretary could easily take some actions on the database based on our gathered data.
At the end of the second week, the secretary came to us with a surprising change in the plan. They had started a new survey and wanted to evaluate it (semi-)automatically, too. The format of the emails was very different (and I think there were some other varieties I don't remember).
We managed to implement these new requirements in the remaining week without much difficulties, by reusing round about half of the existing code. We were able to do this, because the design was so simple (containing no dead freight) that it was very easy to factor out commonalities.
If you want to know more about how this works, you can try to revive Junilu's OO calculator thread

[Junilu Lacar]

Puhleeez, don't revive that thing yet. I want to get it a little more organized. It's a Shu-Ha-Ri salad in it's current state and I'm afraid it'll just get you confused. Feel free to browse through it to see if you can pick something though.

[Ilja Preuss]

Originally posted by Junilu Lacar:
I want to get it a little more organized.

How can we help?

[Junilu Lacar]

I don't really know yet... I still have a few things on my plate that need to be taken care of. I'll get back to you in a few weeks and maybe we can bounce ideas off of each other. It'll be nice to get that thing going again though.

[Ilja Preuss]

Originally posted by Ilja Preuss:
If you want to know more about how this works, you can try to revive Junilu's OO calculator thread

I just realized that this could be understood as discouraging to ask further questions. It certainly wasn't meant this way!
So, please ask questions, object, discuss! That is what we are here for, after all!
Just notice that it probably can't teach you as much as trying things...