what is the best practice for uploading an image to database ?

[tangara goh]

Hi experts,

I need some tips on how to push up an image to a database.
So far, I am told to follow these steps:

1. save the image at  a filePath
2. save the filePath to a database
3. and then write a query to retrieve the image

Now, I also understand that people are using Mongo GridFs to store the images

How do I make the uploaded image go directly to the Mongo database ?

I mean I am not sure how Spring will know which table and column it will go into ?

I have a sample code which specify ${upload.path} and it will copy  from the InputStream and use Files.copy to get the Path and filename.

So do I still need to create a POJO like specifying a fileName and then a Byte[]Image to in order to store the image ?

How should I link the image to another database which specify the location of the image ?

I am really blur and confused how to do the above.  Please let me know how it should be done.


Tks.

[Tim Moores]

Images should not be stored in a DB. File systems are good for storing binary blobs files, and just their metadata should go into the DB.

[Stephan van Hulst]

Tim Moores wrote:Images should not be stored in a DB. File systems are good for storing binary blobs

What makes file systems better for blobs?

[tangara goh]

Tim Moores wrote:Images should not be stored in a DB. File systems are good for storing binary blobs, and just their metadata should go into the DB.

I was told to store encoded Base 64 strings into the DB...but then how do MongoDB comes in ?

Tim,
Your file systems refer to ? It can't be harddisk right ? Doesn't make sense...

[Tim Moores]

Stephan van Hulst wrote:What makes file systems better for blobs?

You're right, I should be precise instead of making a half joke. I fixed my post.

[Tim Moores]

tangara goh wrote:I was told to store encoded Base 64 strings into the DB

DBs can store binary data, so base64 encoding would not really make sense. Ask whoever told you that why that is considered advantageous.

but then how do MongoDB comes in ?

Why do you think it needs to?

Tim, Your file systems refer to ? It can't be harddisk right ? Doesn't make sense...

File systems can work in different ways, but yes - most often it would be on a hard disk.  Why would storing files on a hard disk not make sense?

[Tim Holloway]

tangara goh wrote:
I was told to store encoded Base 64 strings into the DB...

That's a horrible thing to do. Base64 is very space-inefficient. It was designed in part to email binary data between incompatible computers whose byte sizes could be as small as 6 bytes. So typically a Base64 translation of data will be at least twice as big as the original data.

A better solution, as we've said, it to upload the images as independent files to an image directory and store the pathname of the image fine in MongoDB (or whatever SQL/NoSQL database you prefer).

[Stephan van Hulst]

Tim Moores wrote:You're right, I should be precise instead of making a half joke. I fixed my post.

It was a serious question though, I'm not a big DB buff. What reason is there to prefer using a file system over saving the image as a blob in the DB?

[tangara goh]

Tim Holloway wrote:

tangara goh wrote:
I was told to store encoded Base 64 strings into the DB...

That's a horrible thing to do. Base64 is very space-inefficient. It was designed in part to email binary data between incompatible computers whose byte sizes could be as small as 6 bytes. So typically a Base64 translation of data will be at least twice as big as the original data.

Sorry Tim, is base64 and encoded base64 the same thing.  I checked back my message - the person just mentioned base64.

A better solution, as we've said, it to upload the images as independent files to an image directory and store the pathname of the image fine in MongoDB (or whatever SQL/NoSQL database you prefer).

The image directory should be what kind of location ? can you be more specific cos it can't be your local hard drive when you will be hosting the app eventually right ?
So, just to confirm we don't need Mongodb GridFS to store the image ?

Really, I have been surfing entire day just on this topic alone...and I am already behind my deadline ..but I already surrender to my fate....

Sorry for the ranting but hope you guys can advise me...thanks agin.

[Tim Moores]

tangara goh wrote:The image directory should be what kind of location ?

A directory on a file server.

cos it can't be your local hard drive when you will be hosting the app eventually right ?

Well, we don't know what you intend to do with that app :-) But yes, most likely it will run on a host other than some developer's local machine.

just to confirm we don't need Mongodb GridFS to store the image ?

It's really hard to answer this kind of hard-and-fast question. You need just about nothing. It's all a question of what you can or should use. And the recommendation here is to use a file system for files, and a database (possibly MongoDB, since you already seem to use that anyway) for the file's metadata. But you could use GridFS, especially in the circumstances laid out by its documentation: https://docs.mongodb.com/manual/core/gridfs/#when-to-use-gridfs

[Tim Holloway]

When I said base64, it was just short for "base64 encoding", so yes, the two terms are the same thing.

Technically,  you don't have to upload data files to a file server. But in an enterprise environment there are definitely advantages to server-based files over local files. Fileservers tend to get backed up better and are generally more robust.

GridFS does have the ability to store binary files. However, it does so in a different way than regular files get stored, so anything uploaded to GridFS is dependent on MongoDB for access. If you need high-performance file access, the extra slicing, dicing and data management might slow things down unacceptably as well, but that would be something you'd have to determine on a case-by-case basis.

[tangara goh]

just to confirm we don't need Mongodb GridFS to store the image ?

It's really hard to answer this kind of hard-and-fast question. You need just about nothing. It's all a question of what you can or should use. And the recommendation here is to use a file system for files, and a database (possibly MongoDB, since you already seem to use that anyway) for the file's metadata. But you could use GridFS, especially in the circumstances laid out by its documentation: https://docs.mongodb.com/manual/core/gridfs/#when-to-use-gridfs

Hi Tim,

I have seen in my googly that people actually created a POJO class just for the file alone and then they have timestamp, fileName the metadata you mentioned.
Is this necessary ? I thought once I have the file location which is at the fileServer(and this location data will be in MYSQL) and then I just do a relationship mapping to the Image Id in MongoDB what do you think.  But, really now I feel this MongoDB is kind of pointless since the image is already all in the fileServer.... so I would just store everything including the metaData at MYSQL but is metaData really matters ? cos the file will be associated with another Entity (for this case Pet) so why the need to have metedata?

Hope you could clarify on my doubts.

Thanks again for your time.

[Tim Moores]

I have seen in my googly that people actually created a POJO class just for the file alone and then they have timestamp, fileName the metadata you mentioned. Is this necessary?

now I feel this MongoDB is kind of pointless since the image is already all in the fileServer.... so I would just store everything including the metaData at MYSQL but is metaData really matters ? cos the file will be associated with another Entity (for this case Pet) so why the need to have metedata?

Those are implementation details that depend on your circumstances. If you need to keep metadata, you obviously need to store it somewhere.  Whether MySQL or MongoDB is the best place to do so, we can't know. Both can be used, so pick what makes sense for your situation. If you already use one of them, then it doesn't make sense to add the other just for the metadata, because, as I said, both can be used for that. If you're not sure what metadata you'd need, well, maybe you don't need any. That would be unusual in my experience, but it could happen.

[Dave Tolls]

Stephan van Hulst wrote:

Tim Moores wrote:You're right, I should be precise instead of making a half joke. I fixed my post.

It was a serious question though, I'm not a big DB buff. What reason is there to prefer using a file system over saving the image as a blob in the DB?

https://arxiv.org/ftp/cs/papers/0701/0701168.pdf
That's a few years old now, but covers much of the discussion.

I'm not sure much has changed in 14 years, beyond each of the techs involved maybe.

The argument tends to be "files should go in file systems" vs "how do you ensure the data stays in sync".

[tangara goh]

Tim Moores wrote:

I have seen in my googly that people actually created a POJO class just for the file alone and then they have timestamp, fileName the metadata you mentioned. Is this necessary?

now I feel this MongoDB is kind of pointless since the image is already all in the fileServer.... so I would just store everything including the metaData at MYSQL but is metaData really matters ? cos the file will be associated with another Entity (for this case Pet) so why the need to have metedata?

Those are implementation details that depend on your circumstances. If you need to keep metadata, you obviously need to store it somewhere.  Whether MySQL or MongoDB is the best place to do so, we can't know. Both can be used, so pick what makes sense for your situation. If you already use one of them, then it doesn't make sense to add the other just for the metadata, because, as I said, both can be used for that. If you're not sure what metadata you'd need, well, maybe you don't need any. That would be unusual in my experience, but it could happen.

So, just to confirm, if I don't need meta data, I will just have a fixed location where it will be uploaded to ? e.g like bitbucet.
But, since the location is fixed, it should not be necessary to store the file path isn't it ?
I only thing I see as relevant is the different fileImage name...
So, this must be it.

Thank you guys so much for explaining things to me.

[Tim Moores]

Remember that if you're just storing the image in the file system, and it has a fixed name, you'll have no record of who uploaded images, and when, and why - images will just be overwritten. If that is fine for your purposes, then there may be no need for metadata.