So - part of memory is reserved for the JVM to use for classfiles and method areas and String pools and such.
Part of memory is reserved for the JVM to do it's current thinking, this includes the Stacks that local variables live in.
Then there is the heap where all of the objects are just dumped in random order. No one would ever find anything on the heap if there were not references to them kept by the creating objects.
Note that there is some lea way in the Java Language Specification that allows JVM vendors to choose to implement some JVM stuff on the heap if they so choose. That should not matter to you as the programmer.
You might enjoy reading this article about
Not all Variables are created Equal. It describes the layout of the memory in a "less than technical" approach
.