Discussion about Design approach being used in product companies

[Yogesh Gandhi]

Dear Friends,

I need to propose an approach for the following requirement. As far as I have analyzed it doesn't seems possible to me.
I wanted to get a view of experts present here in this forum.

Requirement :

We are a product company. We have a banking product and we customize it for each client.

In Version Control System, we create branch per customer.


We have a parent branch that is currently running on version 3.10
Lets say, we get a customization work here, and we create a child stream/branch out of version 3.10
We work on it for 6 months and deliver the enhancement to the customer

After delivery, Lets say, the core team releases a new version of core i.e. 3.11
Now, In usual scenario, we would have been required to merge all the changes done between 3.10 and 3.11 to our child stream.
But now the company wants, to reduce the merging effort to ZERO. i.e. they want that there should not be any requirement for merging. Only changed files from core team should be taken and replaced into the enhancement stream.

In other product companies, how releases and upgrade go? In Oracle, they release their upgrades without merging, but the kind of product Oracle has is not CUSTOMIZED for each customer. The problem here with our product is that it is customized for each customer.

Please share your thoughts.

Kind of files we have in our code are
1) JSP, JS, Action Class, Action Form, DAO, DTO, Bean

[Winston Gutkowski]

Yogesh Gandhi wrote:Please share your thoughts.

Well, one common model is to have a "core product" whose development is dealt with centrally, and customizations, which are basically collections of "add-ons", whose lifecycle is independent, and perhaps dealt with by a dedicated group for a particular client. Those "add-ons" usually fall into two categories:

Genuinely new functionality.

Existing functionality that behaves slightly differently. You could think of these as you might like "overriding" a method or creating a polymorphic class in Java.

Either way, it helps enormously if your product is developed as sets of interfaces, so portions can be customized by simply providing an alternate implementation.

Another possibility - especially with the maturity of products like Guice - might be to use dependency injection; but I suspect that if customizations get very involved, you could end up with another layer of testing to ensure that your "customized product" works properly (but TBH, I don't know, because I've never used it for something like you intend). However, it is a great - and surprisingly simple - way of substituting classes "on the fly".

Either way, it involves a LOT of design - especially on your core product area - because otherwise any changes to your core classes are likely to have a nasty ripple effect.

The other thing - and this is a sales issue as much as anything - you (or rather, your sales staff) need to understand the limits of what you can customize. No software product can be all things to all people, but sales people want to sell it; and the danger is that they might promise a customization that it was never your intention to deliver, or that ends up taking so damn long that you lose money or customer satisfaction on the venture.

So my advice: Start small.

Offer customizations in specifically targeted areas, and make sure that:
(a) Your core product is ready for them.
(b) Your sales people understand exactly what they are (and, more importantly: what they aren't).
After a few successful ones (and probably a few iterations of changes to them), then think about expanding those areas. By then, you and your company will probably have a lot more expertise, and thus a much better idea of what works and what doesn't.

My ideas, for what it's worth.

Winston

[Yogesh Gandhi]

Thanks for the reply Witson.

Our product is well established in market, and has many versions (one for each client).

Do you mean we need to write the product from scratch?
Our target is to reduce the merging effort to the minimal.

There are lacs of product companies in the world. Is it possible that every product company does merging to release an upgrade to the customer?
There must be some mechanism or design strategy to make this approach work.

[Yogesh Gandhi]

Existing functionality that behaves slightly differently. You could think of these as you might like "overriding" a method or creating a polymorphic class in Java

Here the problem is

Imagine we have an Action class having an execute method.
We extend that class and override the execute method.
In the overridden method, we copy paste the code from parent class and make the changes in the overridden method.
Now, if there happens to be a change in the parent class, that change will never get propagated to the child class's overridden method.

In effect, we have to do merging.

[Winston Gutkowski]

Yogesh Gandhi wrote:Our product is well established in market, and has many versions (one for each client).

So, do you mean that each client has their own version of ALL your product code? Sounds like a nightmare to me.

Do you mean we need to write the product from scratch?

No, but you may well need to do a fair bit of refactoring.

Our target is to reduce the merging effort to the minimal.

You'll have to explain that one to me. Either you have common code or you don't. Obviously, you want to keep customizations as decoupled from your core area as you can, but surely your product (or product modules) have a common core?

There are lacs of product companies in the world.

Sure there are, and most successful ones understand exactly what I said above:
1. A product can't be all things to all people.
2. Customizations (especially big ones) are risk-prone.

There must be some mechanism or design strategy to make this approach work.

Well, I've tried to explain my ideas to you, but if your company already has a "well-established" product with customizations, then the chances are they already have their own procedures in place. The question then is: how much do you want to change them?

Winston

[Winston Gutkowski]

Yogesh Gandhi wrote:In the overridden method, we copy paste the code from parent class and make the changes in the overridden method.

Whoa right there hoss. The second you say "copy/paste", alarm bells go off in my head. That is NOT Object-orientation. It's not even modularity. It's somebody's lazy idea of how to create a new implementation. Sure, it might work for a while, but you'll soon run into problems.

The whole business of customization, whether of a Java class or a software product, is to determine what is different and change (or add, or remove) ONLY THAT. Everything else should continue to use the parent (or common, or core) code; otherwise you're in for a LOT of trouble - and it sounds to me like you might already be there.

Winston

[Yogesh Gandhi]

So, do you mean that each client has their own version of ALL your product code? Sounds like a nightmare to me.

Yes we do have a seperate stream / branch for each client.

Whoa right there hoss. The second you say "copy/paste", alarm bells go off in my head. That is NOT Object-orientation. It's not even modularity. It's somebody's lazy idea of how to create a new implementation. Sure, it might work for a while, but you'll soon run into problems.

Not really, we do not really copy/paste the code. We are thinking on the lines, how to make the existing framework such that there will be least of merging effort.
We have action classes, which do lot of activities including writing into database and many other things. Now, what can we do such that we can use the core files as it is and can easily release the upgrade to the customer. I did thought of many ways, but every approach I can think of has some or the other flaws.

[Winston Gutkowski]

Yogesh Gandhi wrote:Not really, we do not really copy/paste the code. We are thinking on the lines, how to make the existing framework such that there will be least of merging effort.

Well, without knowing an awful lot more about your "exisiting framework", I can only talk in generalities.
1. You need to decouple your custom code (and ONLY your custom code) as much as you possibly can from your core code. It sounds to me like this is what your "action" classes are designed for.
2. Read up on the MVC model - and it sounds like you'll be concentrating, at least to start with, on the V and C portions.
3. Convert as many classes as possible (and from the sound of it, ALL of your "actions") into interfaces; and give them a default implementation class for your core code. Custom code could then use customized implementations of those interfaces, perhaps via the "Decorator" pattern (ie, composition).
4. (optional) Look at Guice, or some other dependency injection framework. You may be surprised what you can do.
5. Test, test, test, test, test.......oh, and test again. Indeed, you might actually do best to design your client code testing harness first.

Winston

PS:
6. Read up on the Bridge and Strategy patterns, because you may find them very useful. In fact, you might want to buy The Bible.

[Yogesh Gandhi]

Thanks Witson. Bridge design pattern does what we want in our product. We need to analyse how we can fit this into our product.

[Yogesh Gandhi]

Let me elaborate things a bit more:

We do have a core stream. For every new customer a new child stream is created and development is done on the child stream.

We have Action class, Action form, DAO, DTO, Bean, JSP, JS

Lets discuss about one action class

AddressAC

it has an execute method

which reads values from Action form and writes into DTO object, then passes that DTO and action(C/R/U/D) to the DAO via Bean class. updates the information into database and displays JSP.

Lets say a customization comes to change a calculation logic.

I am thinking how will we achieve the effect that any changes in core ccan be brought into child stream without the need for merging. What we want is that we should be able to copy paste complete files from core into child stream without the need for merging.

[Mahesh Kedari]

I am not fully aware of your problem. Also I have not worked on framework design of our products, but I'll just explain how our product is handling such scenarios.
We have 2 types of indexing to our customized modules.
1 is for their version and 2nd is for weight.
Certain code in our framework always needs a customization while certain customized code depends upon core version.

As you were talking about Action class,
When we customize these classes we give -1 weight to these classes if we always want a customized class to be used,
else we give some positive number indicating it is free to pick up latest code from core if version index is higher than customized class.
This positive number ( x ) indicates the difference in version decided by project architect.
So if core class version is x points higher than customized class, use core class else use customized class.

These indexes are mentioned in manifest file of customization module.

com.abc.test.MyAction  3.1.0.0  -1
com.abc.test.MySecondAction 3.1.0.0  0.2

Here, MyAction class will always be used from customized module while MySecondClass will be used from customized module till core version is less than 3.3.0.0.

This is not a complete solution for your problem but i hope it helps you for your design strategy.

[Winston Gutkowski]

Yogesh Gandhi wrote:Lets say a customization comes to change a calculation logic.
I am thinking how will we achieve the effect that any changes in core ccan be brought into child stream without the need for merging. What we want is that we should be able to copy paste complete files from core into child stream without the need for merging.

I still don't follow what this "merging" business is all about. It seems to me that if you get this right, any "merging" will be accomplished by the process that creates your jar (or war or ear) file(s). Furthermore, if you keep your custom code separate from your core code, it will be extremely straightforward to provide an "inventory" of the customization so that developers for a particular client know what is customized and what isn't.

Also: if you get it right, a change in a calculation should only require a change in supporting data, it shouldn't affect the code at all. The trick is to write the calculation in such a way that that you can change values (eg, a tax rate) without changing the code.

And I'm still very worried that you keep talking about "copy/paste". DON'T DO IT. A customization should be an organic extension of your core code, not a "copy with some bits changed".

Winston

[Yogesh Gandhi]

May be I am not able to explain it properly.
Let me try it once again...

Merging is, bringing any change done in code to your customized branch, so that you client also gets the fixes for the issues that have been fixed in core.
Lets say

class CoreAddressAC
{
    public ActionForward execute(ActionMapping mapping, ActionForm form,
        HttpServletRequest request, HttpServletResponse response)
        throws Exception {
                  int i = 2 + 3;
                  int k = 2 * 3;
                  // any other code....using these values.
       }
}

class customizedAddressAC extends CoreAddressAC
{

    public ActionForward execute(ActionMapping mapping, ActionForm form,
        HttpServletRequest request, HttpServletResponse response)
        throws Exception {
                  int i = 2 + 3;
                  int k = 2 + 3 + 5;; // This is customized version.

                  // any other code....using these values.
       }


        
}

Now, lets say, the core team decides, that it should be 2/3 in place of 2+3, this change will never come to my customizedAddressAC unless and until I do merging.

I hope I am able to explain my situation.

This is only about JAVA. We don't know, what will we do for JS and JSP.

[E Armitage]

Sounds like you just need to apply common design patterns in your code.

The template method pattern will definitely solve lots of your problems but it's reliance on inheritance means it should be used sparingly. Read up on the following; Open Closed Principle (very important), Liskov Substitution Principle, Dependency Inversion Principle, Component Reuse Principle, Interface Segregation Principle and Least Knowledge Principle.