Great to hear from you Simon. Also thanks for your responses to other questions as well. :-)
Your question kind of has hidden between the lines that G1 was initially developed with the intent of dealing with larger
Java heaps. And, that is true. Our primary focus with G1 initially was to target the larger Java heaps and keeping GC pauses in that area of 200 ms - 500 ms. We also knew there would be tradeoffs in the areas of memory footprint, throughput and CPU usage with a lower GC pause time goal. For instance, on a large Java heap we would expect that Parallel GC versus G1 likely would have higher throughput, higher latency, lower memory footprint and probably use less memory than G1. But, G1 would have lower latency, or a more balanced tradeoff between throughput, latency, footprint and CPU usage.
In more recent updates of JDK 8, and in the under development of JDK 9, we have started to focus on lower latency and smaller Java heaps.
To your specific question on "reasonable expectations for improvement in GC behavior ...", my initial reaction is to look at what was written in Java Performance on tuning the JVM step by step. The general notion from selecting an appropriate GC was the first start with Parallel GC, and if it could meet the application's performance goals as they related to throughput, latency and footprint, to use Parallel GC. But, if latency was too great to tolerate, even after attempting to tune Parallel GC, then the next step suggested step was to move to CMS GC. This is where I would modify that suggestion and also suggest to move to G1 GC. When it comes to tuning G1 GC, tuning it is quite different from the approaches to tuning Parallel GC and even CMS GC. That was one of the motivations we had for content in the Java Performance Companion, i.e. to offer readers the information and "how to" tune G1. I think G1 is easier to tune than CMS GC, but in fairness I am also very familiar G1 GC. I might also mention that because improvements continue to be made to G1, perhaps if you have evaluated G1 at some point time, it may be that some enhancement that has been done since that evaluation, or a future enhancement may address what you have seen.
Might also take this opportunity to say that both Bengt and Monica may have some additional insight to add.