If you're using an HTTP connection to download the images, there's a way to enable caching within it, so you might achieve what you want that way. Every time you start your app, it would go fetch the images, which may not be exactly what you want. But while your app is running, a new fetch should just find the image in the HTTP cache and avoid another fetch over the network.
Otherwise, you should
be able to create an image folder on the SD card to store your downloaded files. Save them off in a format that doesn't require conversion next time you read it back in. Also, if you put an empty file with the filename ".nomedia" in the directory with your image files, Gallery will ignore them. Technically the media scanner will ignore them so Gallery never finds out about them.