• Post Reply Bookmark Topic Watch Topic
  • New Topic

how to make md5 faster ?  RSS feed

 
Ranch Hand
Posts: 72
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I try to get a MD5 digest of a mp3 file(20MB), it takes 150s to do this, how can I get the digest faster?
Any help will be appreciated.
 
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I don't know - I tried your code on my (rather old and slow) machine, with a 59 MB file, and got the digest in under 5 seconds. Of course I had to correct some typos and fill in some blanks. But I'm thinking there's some other problem on your system. Perhaps the filesystem is badly fragmented? Or you've got an NFS drive on a badly overtaxed network? Try measuring the time it takes to just read all the bytes in the file, without calling md.update() (or even getting the digester). Does it take the same amount of time, pretty much? Or is the time being consumed by one of the other actions, most likely the md.update() call? That would give a clue as to whether the problem is with the I/O or something else.
 
Ranch Hand
Posts: 356
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hello,

i wrote JChecksum and compared it to md5sum by the Free Software Foundation. The latter is several times faster: It is exactly as fast as the maximum reading speed of my harddisc.

I did not even find a way to write a Java program that reads a file (without using the data in any way) as fast as md5sum computes the md5sum of the same file. That part must be optimised. (I don't know how much there is to gain by a self-written md5 algorithm.)

Good buffering and using the server-VM rather than the client-VM helps a lot.

The performance of your program is excellent. It is as fast as md5sum by the
Free Software Foundation (or maybe even beats it). Its full potential can not be tested here, because your program computes the md5 as fast as my harddisc can read.

A 250 MB-File took 4 seconds with arifin's program (client and server VM alike), 5 seconds with md5sum and 20 seconds with my program (JChecksum) @AMD 3200+, Maxtor IDE HD, Win2k.

I will adept your buffering to my program.

Conan
[ March 07, 2005: Message edited by: Conan Elvitaro ]
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!