Last line not being written to the file on using BufferedWriter

[ch pravin]

Hello All,

I am facing a weird problem.I have a huge file containing 16000 rows.I am trying to split that file into 40 smaller files(400 lines each). However,when I do so the last line is not being written to the text file. Also,a part of the last but one line is not being written completely.I have no idea what is going wrong here. I tried to print those lines on the console and they print just fine. Any help will be greatly appreciated. I am attaching my code:

		 FileReader fr=new FileReader("/homes/test_output_400.txt");
		 BufferedReader br=new BufferedReader(fr);
		 
		  Class.forName("com.mysql.jdbc.Driver").newInstance();
         	  Connection con=DriverManager.getConnection("jdbc:mysql://localhost/test?holdResultsOpenOverStatementClose=true","root","abc123");
		  
		  String line="";
		  int testCnt=0;
		  int fileCount=0;
		  FileWriter fw=null;
		  BufferedWriter bw=null;
		  
		    while((line=br.readLine())!=null)
	        {
                        //Creates a new file after writing 400 lines
		  	    if(testCnt%400==0)
			   {
			    	fw=null;
			    	bw=null;
				   fileCount++;
				   fw=new FileWriter("/homes/split40test/Test"+fileCount);
				   bw=new BufferedWriter(fw);
			    }
			   
			    bw.write(line);
                          //I tried to introduce sleep for the last two lines,so that they can be written
			    if((testCnt+2)%400==0 || (testCnt+1)%400==0)
			    Thread.sleep(1000);
                           
                           //The last two lines do get printed here
			   System.out.println(line);
			    bw.newLine();
			      testCnt++;
			  
	        }
		 	bw.close();
		 	fw.close();
		 	br.close();

[xsunil kumar]

I will try to debug your program. But i have one more idea. If you are using Linux, then there is a command to split the file. Exactly i do not remember the command. You can get that on google.

-Sunil

[Stefaan Dutry]

when you create a BufferedWriter using a FileWriter you shouldn't close the FileWriter anymore.
Closing the BufferedWriter automaticaly closes the Filewriter too then (decorator pattern).

If you read Buffered Writer javadoc for close() method you will see why it could fail when you close the FileWriter ( see the description).

If you have more questions,
please ask.

[ch pravin]

xsunil kumar wrote:I will try to debug your program. But i have one more idea. If you are using Linux, then there is a command to split the file. Exactly i do not remember the command. You can get that on google.

-Sunil

Thanks for the reply. I am using windows though

[ch pravin]

Stefaan Dutry wrote:when you create a BufferedWriter using a FileWriter you shouldn't close the FileWriter anymore.
Closing the BufferedWriter automaticaly closes the Filewriter too then (decorator pattern).

If you read Buffered Writer javadoc for close() method you will see why it could fail when you close the FileWriter ( see the description).

If you have more questions,
please ask.

Thanks for the reply.However, the problem persists!

[Stefaan Dutry]

Although the javadoc states that the close() method flushes the stream, you can try to flush it first yourself before closing.
BufferedWriter javadoc flush() method

Can you tell me if that helps?
Don't see an immediate problem otherwise realy.

[Stefaan Dutry]

think i spotted the real problem now

you're only closing and thereby flushing the buffered writer once and not each time you make a new one => possibly having characters in your buffer not yet written when you make a new one and losing them

[Campbell Ritchie]

More seriously, you are setting the writers to null. Don't do that; when you have finished with a writer (or a reader) you should close it, preferably in a finally block.
Why don't you use the more modern tools, Scanner and Formatter?
Why on earth are you mixing database connectivity and text file writing in the same method?
I don't think your thread.sleep call helps at all. It probably does nothing except delay execution.

[ch pravin]

Stefaan Dutry wrote:think i spotted the real problem now

you're only closing and thereby flushing the buffered writer once and not each time you make a new one => possibly having characters in your buffer not yet written when you make a new one and losing them

Thanks! That worked

[ch pravin]

Campbell Ritchie wrote:More seriously, you are setting the writers to null. Don't do that; when you have finished with a writer (or a reader) you should close it, preferably in a finally block.
Why don't you use the more modern tools, Scanner and Formatter?
Why on earth are you mixing database connectivity and text file writing in the same method?
I don't think your thread.sleep call helps at all. It probably does nothing except delay execution.

Thanks for the reply.It was really insightful.I just wanted to know if you could point me to some resources from where I can learn the best programming practices? I had no idea that mixing database connectivity and text file writing in the same method is a bad programming practice.

[Jesper de Jong]

In general, methods should have a single purpose or single responsibility (see Single responsibility principle - this not only applies to classes, as the article says, but to methods as well). There are a number of good books on object oriented design. One of the most famous books is Clean Code: A Handbook of Agile Software Craftsmanship by Robert C. Martin ("Uncle Bob").

When you do JDBC stuff and writing to a file in one method, then you're mixing two APIs that both require a fair amount of boiler plate code and exception handling, and putting all that stuff in one method will make it complicated and hard to follow.

It's not so that it should not absolutely never be done that way, but there are ways to structure your code differently so that it is easier to read and maintain.

[ch pravin]

Jesper de Jong wrote:In general, methods should have a single purpose or single responsibility (see Single responsibility principle - this not only applies to classes, as the article says, but to methods as well). There are a number of good books on object oriented design. One of the most famous books is Clean Code: A Handbook of Agile Software Craftsmanship by Robert C. Martin ("Uncle Bob").

When you do JDBC stuff and writing to a file in one method, then you're mixing two APIs that both require a fair amount of boiler plate code and exception handling, and putting all that stuff in one method will make it complicated and hard to follow.

It's not so that it should not absolutely never be done that way, but there are ways to structure your code differently so that it is easier to read and maintain.

Thank you very much. I'll look into the book.