Without going on and on with the details, do you feel that GWT is a good solution for adding AJAX into an existing application? It seems like it'd be great from starting from the ground up.
If you're interested in our AJAX problem, we tried Dojo for a month but could never get it working up to our standards, just hogged way too much memory when displaying thousands of nodes of a tree. So we continued to hack at our CSS solution and made it dynamically load, and that works reasonably well. Now I'm implementing some Yahoo User Interface (yui) on some other parts of the site and it's working great.
As to the high-level question, how GWT fits into an existing application, I'll see if I can explain.
GWT uses a concept of a panel, and panels can contain other panels and widgets. This is similar to Swing. In most cases you add objects to the pre-defined RootPanel, which corresponds to the BODY element in the page, but this doesn't help with existing applications, or when you need to mix GWT with coded HTML.
In those cases you can attach panels and widgets to an HTML element, based on its id value. So you would put an empty DIV element somewhere on your page, then create panels/widgets and add them to that div.
In your specific case it sounds like you could easily add GWT by simply targeting the DIV where the tree widget needs to appear. The problem, based on your post, seems to be that the tree has so many items that it is causing some performance issues.
So... to answer the question, I don't know if GWT will work in your specific case. You may implement it just to find that you are running into the same performance issues that you did with Dojo.
On the flip side of that coin, you can easily make your own widgets that are targeted to your specific performance issue. That and the fact that performance enhancements are constantly being added, so maybe it will just work out-of-the-box.
Sorry for the overly long answer, but if you have some extra bandwidth it might be worth trying. You might find that GWT solves the issue in a very maintainable way.
It took me about two weeks to get past the idea of a Java-to-JS compiler. After that I stopped thinking about the underlying JS code, and only thought about the Java.