First, if you are using Windows Task Manager to look at memory usage, be sure to use the "VM Size" column. The "Mem Usage" column can be confusing, as it relates to physical memory.
Second, even when looking at VM Size column, this is telling you about all memory used by the JVM. The
Java heap is only one part of that memory. There is additional memory for the JVM's implementation code, loaded Java classes and a variety of other non-heap resources.
In your particular application, I would be surprised if Java was actually increasing the heap size from the initial 32 Mbytes. If it is, then I suspect that either your application is more complicated than you suggest or you have a leak.
To find out whether your application really is increasing the heap size, try setting the maximum heap to 32 Mbytes as well. If the application runs OK and the VM Size is similar to before, then any observed memory consumption increase is due to things other than Java heap.