The java.util.Properties class (assuming that is what you used) extends a HashMap and should by itself serve as a cache of sorts. Are you referring to the load method ? -
http://download.oracle.com/javase/6/docs/api/java/util/Properties.html#load(java.io.Reader)
File IO is an expensive operation. You can look at the last modified time of a file and cache that too. If the last modified time has not changed, do not reload the file.
You can also configure a Timer and TimerTask to poll the file ever X seconds and perform this operation. That will reduce the overhead even more, but will introduce a latency of X seconds for every property change.