Win a copy of Head First Agile this week in the Agile forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

nio download problem  RSS feed

 
Samuel Mendenhall
Greenhorn
Posts: 18
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am using nio to download a file given from a web address. I have not conducted extensive testing but my code works for text files and zip files, but NOT exe or bin files. I assume it works for all text and no binary (although zip is binary right?). The weird thing is, I set up tomcat, and I can download a zip, open the exe in it and it works, but if I extract the exe and try to download via tomcat, it downloads the correct number of bytes, but the exe is not valid, won't open.

I think my problem exists in the GET header I am passing the socket, maybe I need to define the mime type I want to accept? Any body can pass me some help, I'd be greatly appreciative.

Here is the GET header as descibed below also



This is a full "working" example, the exe I am downloading I choose at random, choose another if you'd like.


[ November 03, 2005: Message edited by: Joe Ess ]
 
Samuel Mendenhall
Greenhorn
Posts: 18
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I think I partially figured out the problem. With the GET header I posted in the initial topic, I examined the response in ultraedit and the server said no host was defined, so here is my new GET



Which seems to return the actual binary.....with the response Prepended.

So now my question is this, how do I rip the response out of the binary, while keeping the binary intact.
 
Samuel Mendenhall
Greenhorn
Posts: 18
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
New problem....

The above code is run in a thread, when the jvm exits out, and I try to delete the newly written file, Windows won't allow me, There is still a handle somewhere to the file.

I am closing the fileChannel however, what else should I do so there are no more handles/hooks/whatever attached to the file.

I do NOT have this problem with regular io, only nio.
 
Joe Ess
Bartender
Posts: 9429
12
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Samuel Mendenhall:

I do NOT have this problem with regular io, only nio.


Any reason you aren't using regular io?
Or for that matter, using sockets instead of URL/URLConnection?
 
Samuel Mendenhall
Greenhorn
Posts: 18
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I might be misled, so please call me on it if I am; however I was led to believe that nio is faster because it uses native system calls, it is non blocking, and I am using the sockets becuase they should have less overhead than regular io.

From visual observance, nio seems to be much quicker than regular io.
 
Samuel Mendenhall
Greenhorn
Posts: 18
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Additionally, I fixed my original problem of parsing out the HTTP response; I had to read and discard the bytes until ODOAODOA which is two newlines, which signifies the start of the content.
 
Joe Ess
Bartender
Posts: 9429
12
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You were misled. Regular IO and sockets were reimplemented using Channels when NIO was introduced so throughput differences should be minimal. Where they differ is in feature sets. NIO can be nonblocking (which is good for high-performance servers, doesn't do much for clients) and has other features like file locking. Have a look at Top Ten New Things in NIO for more on NIO.
Have a look at this example of downloading from a URL. You'll have to change it to use a Stream to handle binary files but it should give you an idea of how simple it is to use the existing classes rather reinventing the wheel (remember, being a productive programmer is as important as writing efficient code).
 
Samuel Mendenhall
Greenhorn
Posts: 18
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Isn't there a performance hit on using streams instead of buffers?
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
NIO can be faster than IO, but in many situations there's not an appreciable difference, and sometimes IO is faster. Using buffers can indeed be faster than streams, as with buffers dont need to spend as much time passing bytes in and out of the JVM memory space. Especially if your Java program doesn't insist on actually looking at the individual bytes itself. You can often get really good performance from transferFrom() and transferTo() in FileChannel. To use this you need to know the number of bytes being transferred though - you should be able to get this from the response header. Note that you need to check the return value of these methods and put them in a loop to ensure that you've transferred everything you requested.

As far as NIO supporting nonblocking operations - that's true, but from the way you're using it I don't see any particular benefit to that. You've got a single thread dedicated to downloading one particular file, and if some of the content is not yet available from the server, there's nothing else for your program to do other than try again. You might as well just use blocking mode. If at some point in the future you find you have too many threads and want to instead have one thread handle multiple IO operations, then look into using nonblocking IO in conjunction with a Selector. For now though that seems like excessive complexity, in my opinion.
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
As far as being able to delete the file - you say you're closing the FileChannel, but your code above definitely does not close the FileChannel. Has the code been modified? And are you sure the FileChannel is really closed? From the way you close the other channel, it looks like if an error were to occur, you would have no evidence of it - the exception is ignored. I recommend at least putting a printStackTrace() or some more sophisticated logging into the catch block.

As an experiment, you might also insert a brief Thread.sleep() after the file close, before the delete attempt. It seems that on some [defective] systems, a file may not be fully "closed" yet when close() returns. See if adding a delay makes a difference.
 
Samuel Mendenhall
Greenhorn
Posts: 18
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you for the response.

I will be downloading potentially very large files, 100mb, 500, an arbitrarily large number and I will have multiples threads connecting to servers.


Here is some working code so far, it doesn't seem too complex to me...yet.


 
Joe Ess
Bartender
Posts: 9429
12
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
One quick word, Reverse these two:

If fileChannel is null you'll get a NullPointerException. Same with the other if statement on fileChannel.
 
Samuel Mendenhall
Greenhorn
Posts: 18
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'll do that, I can probably drop the .isOpen() and only test for !=null. I put the .isOpen in as a longshot becuase of the channel not closing on the file.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!