Bytecode enhancement controversy

[Tony Morris]

I have played a little with Hibernate, and I have read a few articles on JDO so I don't consider myself well-informed in the area of O/R Mapping.
However, I have a question regarding the sometimes subtle implications that I read regarding controversy about bytecode enhancement as the solution proposed by JDO. Sure this is different to how a mature tool like Hibernate achieves its goal, but I haven't seen any strong arguments as to why this is a bad thing.

I'm looking for arguments for or against bytecode enhancement and why this causes some general lack of acceptance of JDO. I personally can't think of anything with any substance.
- The source file doesn't correspond to the class file (so what?)
- Development tools that auto-generate class files from source files would wipe out enhancement (again, so what? use a better tool)

I might also point out (as I feel obliged to do on a public forum ) that I'm not looking to fuel a debate - just to gain more insight such that I can form my own informed opinion and determine whether or not this really is an issue worth considering.
[ August 31, 2004: Message edited by: Tony Morris ]

[Erik Bengtson]

I don't think this controversy exists, no more. AOP uses the same approach and is well accepted.

JDO does not mandate bytecode enhancement, but the PersistenceCapable interface implementation. Most of JDO implementations provide bytecode enhancement before runtime, but you can also enhance classes at runtime.

[Christian Bauer]

The question is if bytecode enhancement is neccessary for the things you want to do. Hibernate 2.1.x doesn't need any bytecode enhancement at build, all can be done at runtime, invisible to the developer. Hibernate3 alpha has a (very) optional feature that can only be used with an additional instrumentation step.

It's funny that you metion the JDO spec, where half of the pages are for a bytecode standardization. In my opinion, there shouldn't be a discussion if that is good or bad (I think you can guess what I'd say), but if it is neccessary for what you get.

[Gavin King]

I don't think that people have really argued that bytecode instrumentation is a priori a Bad Thing.

I personally very much prefer tools which do not get involved in my build cycle, because of my personal style of development, which is eclipse-style incremental compile and TDD-style fast code/test/debug cycles. However, it would be theoretically possible to have a bytecode processor which integrated with the IDE's incremental compile (I'm not sure if any existing JDO enhancers to it that way).

Certainly, there are usecases (eg. AOP) for which bytecode instrumentation is probably the right decision.

The debate is more about whether a proxy-based architecture or a field-interception based architecture is the right way to implement *persistence* (there are arguments on both sides) and, more broadly, about whether the JDO spec as written in 1.0, and even in 2.0, simply specifies too much about how an ORM solution should be *implemented*, as opposed to how it should appear to users. We don't believe that it is necessary for a spec to mention anything about this kind of thing, and that it should be completely up to the implementor to decide how to do these things. In JDO2, bytecode instrumentation is strictly speaking optional, however the spec still spends two chapters on the subject, as well as going into many other details which seem to unnecessarily restrict implementation strategies.

Fortunately, the EJB3 spec remedies these problems, by defining persistence behavior from the point of view of the user, without getting bogged down in implementation details.

This is of course, all just my opinion, and some other experienced people in the field disagree.

By the way, if the source file did not correspond to the class file, that could potentially make debugging *incredibly* difficult; however, the JDO folks tell me that it is possible to implement the enhancer so that this is not a huge problem (I suppose the enhancer leaves the line numbers intact in the class file). I'm still not sure exactly how you are supposed step into the JDO implementation code from your business logic, but I guess since all credible JDO implementations are closed-source, you can't do that anyway ;-)

[Gavin King]

>>JDO does not mandate bytecode enhancement, but the PersistenceCapable interface implementation.<<

Build time bytecode or sourcecode enhancement are the ONLY realistic strategies for implementing PersistenceCapable and all credible JDO implementations use buildtime bytecode enhancement.

Actually, I should say that a third option is instrumentation at classloading time, which can be done if you are able to control the application server classloader (for example I can do it in JBoss). However there are many issues with the assumption that you control the classloader.

[Pradeep bhatt]

Hibernate 2.1.x doesn't need any bytecode enhancement at build, all can be done at runtime, invisible to the developer.

Hello Christian/Gavin

Doesn't runtime enhancement affect performance? :roll:

[Gavin King]

By "runtime", we mean startup time. So yes, there is a performance hit at startup. But no, there is no performance cost in the running system.

[Alexandru Popescu]

I think one issue against runtime weaving/bytecode engineering is that the debugging is more difficult (my 2cent).

[Pradeep bhatt]

Originally posted by Ali Pope:
I think one issue against runtime weaving/bytecode engineering is that the debugging is more difficult (my 2cent).

Is it possible to enable Hibernate log messages? :roll: This may be useful.

[Alexandru Popescu]

Is it possible to enable Hibernate log messages?

AFAIK Hibernate is able to log all SQL (hibernate.show_sql=true). On the other hand Hibernate use log4j so basically you can log anything.

[Karthik Guru]

Originally posted by Erik Bengtson:
I don't think this controversy exists, no more. AOP uses the same approach and is well accepted.

I had a question here. I saw examples where the queries fired by cmp were being inspected by writing aspects, i guess on the call to the prepared statement etc. Now if i do this say with weblogic. That would mean that I w'd have to run the aspect compiler on say weblogic.jar that results in a modified jar that in turn has to be supplied to the customer?

not sure of my understanding is correct.

Would BEA agree to this? but I do agree that i w'd probably sniff the cmp queries during development to tune applcn performance and not during deployment. I mean I would'nt need these aspects during deployment.

[Alexandru Popescu]

I saw examples where the queries fired by cmp were being inspected by writing aspects, i guess on the call to the prepared statement etc.

For logging queries you can use Hibernate options or even involve P6Spy library.

About weaving the weblogic.jar I agree it doesn't sound too nice. But talking about querying the weaving must be done upon the JDBC driver - which afaik is not part of weblogic.jar.
However, if I am not (very) wrong, using for the pieces of advices call pointcuts will not require this - (weaving will happen only on your classes).

[Eric Samson]

Originally posted by Gavin King:
I personally very much prefer tools which do not get involved in my build cycle, because of my personal style of development, which is eclipse-style incremental compile and TDD-style fast code/test/debug cycles. However, it would be theoretically possible to have a bytecode processor which integrated with the IDE's incremental compile (I'm not sure if any existing JDO enhancers to it that way).

Yes, there are, at least LiDO.

Best Regards, Eric.

[Eric Samson]

Originally posted by Gavin King:
By the way, if the source file did not correspond to the class file, that could potentially make debugging *incredibly* difficult; however, the JDO folks tell me that it is possible to implement the enhancer so that this is not a huge problem (I suppose the enhancer leaves the line numbers intact in the class file).

Credible byte-code enhancers simply respect the JPDA standard (Java Profiling Debugging). Then debugging becomes *incredibly* as standard as usually. Please note that most JDO byte-code enhancers rely on well known generic open source byte-code enhancers like the one from Apache (BCEL) or ObjectWeb (ASM).

Originally posted by Gavin King:

I'm still not sure exactly how you are supposed step into the JDO implementation code from your business logic, but I guess since all credible JDO implementations are closed-source, you can't do that anyway ;-)

A typical application programmer would certainly never want to step into the technical source code of an O/R mapping tool. Unless there are bugs he thinks he can fix by himself very quickly.

Best Regards, Eric.

[Christian Bauer]

A typical application programmer would certainly never want to step into the technical source code of an O/R mapping tool. Unless there are bugs he thinks he can fix by himself very quickly.

Which happens very often. We get many patches from many developers for problems they encountered in a special case. We have many creative examples of using and extending Hibernate written by "typical" Java developers (who don't have to be experts with bytecode enhancer/VM knowledge); things you can do with Hibernate we did never think about. It's one of the most imortant aspects of good middleware that clean and documented source is available.
[ September 01, 2004: Message edited by: Christian Bauer ]

[Gavin King]

It is absolutely common to want to step into the sourcecode of infrastructure you rely upon when resolving tricky problems. I remember trying to work with WebSphere CMP back in the day, and how utterly impossible that was, since I had no way of figuring out what it was actually doing.

Relying upon closed-source infrastructure is incredibly risky when you think about it; that's why the whole industry is moving to opensource solutions. (My opionion, of course, but borne out by statistics for application server and operating system market share.)

[Peter Reinhardt]

I think one problem of Byte Code modification (especially when used with Hibernate) is that modified classes can't be serialized, i.e. if you use the hibernate classes as transfer object in a distributed environment.

The reason for that is, that a client doesn't know anything about the dynamic generated classes on the server.

Peter

[Pradeep bhatt]

i.e. if you use the hibernate classes as transfer object in a distributed environment.

That is really bad. Thanks for the useful info .

[Gavin King]

I think one problem of Byte Code modification (especially when used with Hibernate) is that modified classes can't be serialized, i.e. if you use the hibernate classes as transfer object in a distributed environment.

This is not true, of course.

I'm not sure where you got that information from, since it is a major selling feature of Hibernate that persistent objects can be detached, serialized, and then later reattached.

The reason for that is, that a client doesn't know anything about the dynamic generated classes on the server.

Hibernate is able to regenerate the dynamic proxy classes on the client side.

[somkiat puisungnoen]

Hibernate is able to regenerate the dynamic proxy classes on the client side.

It's seem like stup/skelon of RMI or Home/Remote of EJB or not ??

[Gavin King]

It uses writeReplace() / readResolve() of standard Java serialization. You really don't need to worry about this stuff, it is completely transparent to the user.

If you are interested, look at the net.sf.hibernate.proxy package.
[ September 02, 2004: Message edited by: Gavin King ]

[Pradeep bhatt]

This is not true, of course.

I'm not sure where you got that information from, since it is a major selling feature of Hibernate that persistent objects can be detached, serialized, and then later reattached.

Thanks for the clarification.