• Post Reply Bookmark Topic Watch Topic
  • New Topic

File copy from input stream is not complete  RSS feed

 
murali jackson
Ranch Hand
Posts: 52
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi All,
I have written a small program to copy the file from input stream and write the same to a new file.
Below is my program which works fine. But the thing is its too slow, it takes more time to complete. It was copying one byte at a time thats why it was slow.
So I have modified the program as the second program mentioned below . I have declared byte array and used that as a buffer size to read from the inputstream.
I thought it will write 1024 bytes at a time and the program would be faster, but it stops in the middle and when i open the outputfile it was just having below as content.



Please advise what is actually going wrong in my modified program... Thanks !!!

Working Program but Slow :



Not working Code :



 
Henry Wong
author
Sheriff
Posts: 23280
125
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The behavior of the read() method, of the before and after version, are different. And hence, your write() methods have to change too.

Basically, all your new code does is write the number of bytes that are read (as ASCII codes), and not the bytes that are read.

Henry
 
Norm Radder
Ranch Foreman
Posts: 2222
28
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Using proper variable names could have helped here.  Instead of the name: i use the name: nbrBytesRead
Instead of b, use bytesReadBuffer
 
Campbell Ritchie
Sheriff
Posts: 55351
157
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Why are you using such low‑level code? My experiences with read() methods have been uniformly bad; I look on them as nothing but an annoyance. Use readLine() and writeLine(...) methods instead.
Also, why are you using input streams and output streams for text files? For text files use Readers and Writers. Then use a BufferedReader and a BufferedWriter.
Do you need to write flush() and close(). I thought close would call flush first.
What are you using the byte[] for?
Why are you using the close method when you could have used try‑with‑resources.
Go through the Java™ Tutorials and see how it ought to be done.

There is a slightly different method with Scanners and Formatters.
 
Campbell Ritchie
Sheriff
Posts: 55351
157
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have given you the wrong name for one of the method; sorry; it was by mistake.
Read about the read(byte[]) method carefully.
 
murali jackson
Ranch Hand
Posts: 52
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks All for the advise !!!
It was very helpfull ... I will modify the code as per the suggestions provided ...

 
Campbell Ritchie
Sheriff
Posts: 55351
157
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
murali jackson wrote:Thanks All . . . It was very helpfull . . .
That's a pleasure. Do show us the improved version.
 
murali jackson
Ranch Hand
Posts: 52
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Campbell Ritchie wrote:That's a pleasure. Do show us the improved version.


Hi,

I have modified the code as below to use the bufferedreader and buffered writer instead of Inputstream.
The program is working but im seeing below issues . Please advise ...

1. The blank spaces in between the lines are missing in the output file.
1. File is only 98 % copied, always the final text in the file is incomplete ("Testing" in the source file is copied as "esting")

Example ..

Source file has below contents,


My output file is written as below



But if i use fileinputstream and fileoutputstream methods, the file copying works perfectly without issues. I have posted both the programs below.
Please throw some light on this.




But the below code works perfectly in all scenarios ..


 
murali jackson
Ranch Hand
Posts: 52
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi ,

I have understood what mistake im doing,

I modified the program to below and all is well now !!! Thanks again for the solution !!!

 
murali jackson
Ranch Hand
Posts: 52
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

Please advice whether my modified program above is correct or it still needs some improvement ?? Thanks !!!
 
Carey Brown
Saloon Keeper
Posts: 2846
43
Eclipse IDE Firefox Browser Java MySQL Database VI Editor Windows
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
writer.close() will do a flush.
 
Campbell Ritchie
Sheriff
Posts: 55351
157
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
murali jackson wrote:. . .. . .
Nonononononononono.

What will happen is that you read the first character in the line: now are you going to read the whole of the line, or only 99% of it. That must be why you are changing “Testing” to “esting”. I told you, the read() method is too low‑level to be any good. I am not at all convinced you need the byte[]s at all. Use the high‑level methods.
Yesyesyesyesyesyes. Well done That will work. By using the strange syntax in line 20, you make sure you only read each line once, so you get the whole of the line. I notice you have also got advance→dvance and begin→egin, so it does appear that the read() method is consuming the first letter of each line. Line 20 (as I said) makes sure you don't read two lines per loop; you would miss alternate lines and there is a risk that the last line in the output would read “null”.
Find out about try with resources as I told you yesterday, which obviates the need to close the readers and writers. If you write close(), you probably don't need to write flush() because close() calls flush() anyway.

The alternative with Scanner and Formatters would look like this:-Note that a Scanner can take a Path as its constructor parameter, but a Formatter can't. I used the Formatter(String) constructor. More details in this old post. That class looks so simple and good that I added documentation comments and shall keep it for future reference.

Please have a look at the formatting in your post. Your line 20 would be a lot better with a bit of space between tokens, like this:-The second version was correctly spaced. You have however too many empty lines in that method.
 
Norm Radder
Ranch Foreman
Posts: 2222
28
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Note: If you want an exact copy of the file, this code might change the lineend characters.  Windows will recognize several different combinations of newline and carriage return as the end of a line.
The output from this code may have a different end of line character that the input.

Using read and write with bytes will keep the output the same as the input.
 
Carey Brown
Saloon Keeper
Posts: 2846
43
Eclipse IDE Firefox Browser Java MySQL Database VI Editor Windows
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
New IO (nio) has another way to copy files using channels. The idea here is to push the loop down to the operating system so that it can be run as close to the hardware as possible, ie "more efficient".

 
murali jackson
Ranch Hand
Posts: 52
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi All,

Thanks again for the detailed explanation !!!
It was really helpful

I have one last doubt on this topic .. what if i need to send this file to another method which is in a different class or if its in a different machine altogether like a web service.
Basically like a client server model. Client requests for a file and my server program creates the file and writes contents into the file and sends it back to client.

I have written below sample program based upon my earlier modified program. In My Server Class, I have created file and written some sample content inside it using buffered writer.
After that I have returned the buffered writer to my calling program. But after that i dont know how to process the buffered writer in my calling program to save the file.




What im trying to learn is , my server method should be in such a way that any class can call this method and get the file and save it .
like below ... please advise...





 
Carey Brown
Saloon Keeper
Posts: 2846
43
Eclipse IDE Firefox Browser Java MySQL Database VI Editor Windows
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You can send a file name but if your server and client are on two different machines that won't work. Otherwise you have to send all the bytes of the file. When doing this the typical approach is to load the file into a byte[] that can hold the whole file, then send the length of the array in the first four bytes followed by the buffer contents. On the client side you then read four bytes to get the file length and then loop till you've read that many bytes from the server. One way to do this is with serialization.

You can't send the file directly nor can you send the BufferedWriter, those will be meaningless to the client.?

How are the client and server supposed to communicate with each other? Sockets? HTTP? File name if they're on the same machine?
 
murali jackson
Ranch Hand
Posts: 52
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Carey Brown wrote:You can send a file name but if your server and client are on two different machines that won't work. Otherwise you have to send all the bytes of the file. When doing this the typical approach is to load the file into a byte[] that can hold the whole file, then send the length of the array in the first four bytes followed by the buffer contents.


Hi Carey,

This is exactly what i want to do, but dont know how to load the content of the file to  byte[] . Could you please share some sample code or tell me what class and methods i need to use to load the file to byte[].

Carey Brown wrote:

How are the client and server supposed to communicate with each other? Sockets? HTTP? File name if they're on the same machine?


I am thinking of using this method as a Webservice operation. So i thing the protocol would be SOAP.  Whenever a client makes a request to this service method.
It will send the byte[] as response and Client will process the byte[] to get the file. Thanks !!!



 
Carey Brown
Saloon Keeper
Posts: 2846
43
Eclipse IDE Firefox Browser Java MySQL Database VI Editor Windows
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
murali jackson wrote:Could you please share some sample code or tell me what class and methods i need to use to load the file to byte[].

This is the jist. Needs some error handling.
 
murali jackson
Ranch Hand
Posts: 52
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for providing the code Carey !!!
I will use this and post my update program here ...
 
Norm Radder
Ranch Foreman
Posts: 2222
28
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Will the file contain binary data? For example line end chararcters. Can SOAP handle binary data? The byte array could be Base64 encoded to ensure no binary data.
 
Piet Souris
Rancher
Posts: 1943
66
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You can simplify this code somewhat by using Files.readAllBytes,  see the Files api.
 
murali jackson
Ranch Hand
Posts: 52
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Norm Radder wrote:Will the file contain binary data? For example line end chararcters.


Hi Norm,

The file is going to be a text file or a pdf, so i guess it might have line end character.

Norm Radder wrote: Can SOAP handle binary data? The byte array could be Base64 encoded to ensure no binary data.


Sorry , I am not sure on this as im just starting to learn java webservices.

Also I have one doubt on the sample program provided...



This is the file size of the pdf file in my hard drive. Carey told that the first four bytes will have the file length and the rest will be the actual file content. But here the entire byte array is having the length of the file.
I am finding it difficult to understand where is the first four byte which has the length of the file and the rest of the bytes which contains the data.
Sorry if my question is too silly .. I just could not visualize how the byte array is used by the program... Please explain.
 
Carey Brown
Saloon Keeper
Posts: 2846
43
Eclipse IDE Firefox Browser Java MySQL Database VI Editor Windows
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This is *a* way to do it. As mentioned you might need to use base64 which, I believe, handles the length for you(?). Serialization can also handle the length for you.
 
murali jackson
Ranch Hand
Posts: 52
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Carey,

Sorry for the late response..
Thanks for the sample program ... im trying to visualize the working of the code.. I will let you know if i still have any doubts on it..
Thanks Again ...
 
Campbell Ritchie
Sheriff
Posts: 55351
157
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
What would be wrong with reading the entire text into a single String, creating a byte[] of the appropriate size, and using String#getBytes()? Apart from the problem of changing line ends, as noted in the documentation comment of my FileCopier class?
 
Carey Brown
Saloon Keeper
Posts: 2846
43
Eclipse IDE Firefox Browser Java MySQL Database VI Editor Windows
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Campbell Ritchie wrote:What would be wrong with reading the entire text into a single String, creating a byte[] of the appropriate size, and using String#getBytes()? Apart from the problem of changing line ends, as noted in the documentation comment of my FileCopier class?

I'm not convinced, based on Murali's comments, that we are only dealing with text files. It could be, say, an image file. Although in Muralis's code the file that is being opened is a ".txt" file, I'm assuming that was only for testing purposes.

Murali, please clarify.
 
murali jackson
Ranch Hand
Posts: 52
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Carey and Campbell,

My Program can process other file types also besides text like Jar file, Image file etc.. Thanks !!!
 
Norm Radder
Ranch Foreman
Posts: 2222
28
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Can you state what you are trying to do again?  The last bits of code were for a small, specific part of the project.  Can you post a higher level description of what you are trying to do?
 
murali jackson
Ranch Hand
Posts: 52
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Norm Radder wrote:Can you state what you are trying to do again?  The last bits of code were for a small, specific part of the project.  Can you post a higher level description of what you are trying to do?


Hi Norm Radder,

I am trying to build a client server model where the client will request a file from the server, it can be any of the following types.

1. txt
2. word
3. pdf
4. excel
5. Jar
6. Jpeg , png.

The server will send the requested file in the form of byte[] and the client will process and save the file .

Please let me know if you need more information.
 
Norm Radder
Ranch Foreman
Posts: 2222
28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
So are these the steps the programs will take?

the client will request a file from the server
server will send the requested file
client will process
and save the file

Do you have a protocol for the requests the client sends to the server
and for how the server responds to the client?

  in the form of byte[]

Specifying how the files will be sent may be too detailed for this level of discussion. 
Ultimately the file's bytes will be sent.
 
Ron McLeod
Saloon Keeper
Posts: 1562
222
Android Angular Framework Eclipse IDE Java Linux MySQL Database Redhat TypeScript
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Norm Radder wrote:Do you have a protocol for the requests the client sends to the server and for how the server responds to the client?


Earlier in this thread ...
Carey Brown wrote:How are the client and server supposed to communicate with each other? Sockets? HTTP? File name if they're on the same machine?

murali jackson wrote:I am thinking of using this method as a Webservice operation. So i thing the protocol would be SOAP.  Whenever a client makes a request to this service method.
It will send the byte[] as response and Client will process the byte[] to get the file. Thanks !!!
 
Norm Radder
Ranch Foreman
Posts: 2222
28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Whoops, my bad.  I don't know SOAP so it didn't mean anything at the time.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!