This week's book giveaway is in the JavaScript forum.
We're giving away four copies of Cross-Platform Desktop Applications: Using Node, Electron, and NW.js and have Paul Jensen on-line!
See this thread for details.
Win a copy of Cross-Platform Desktop Applications: Using Node, Electron, and NW.js this week in the JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

using BufferedReader.readLine()  RSS feed

 
kat basobas
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Im a beginner in Java. Currently I have an existing code that uploads file but when I upload large files like 4MB it takes hours to complete the process.
Can someone tell me what's wrong with my code or is there another way to tune it to lessen uploading hours.

Here is part of the code:

while (result.next())
{

byte[] bytes = result.getBytes(1);
try
{
ByteArrayInputStream ipt = new ByteArrayInputStream(bytes);
InputStreamReader isr = new InputStreamReader(ipt);
BufferedReader br = new BufferedReader(isr);
String line = null;

String ins = "INSERT INTO EL_UPLOAD_DETAILS (REFERENCE_NO, SEQ_NO, FILE_DETAILS, FILE_TYPE, FIELD_GROUP, FIELD_COLUMN, ATTRIBUTE299, ATTRIBUTE300) VALUES (?,?,?,?,?,?,?,?)";
PreparedStatement ppsmt = conn.prepareStatement(ins);

while ( (line = br.readLine()) != null)
{
if (line.length() > 1)
{
String repk = line.substring(0,2);
if (repk.equals("00"))
//if (repk == "00")
kn2++;
kn++;
System.out.println("im here na @ 2");
if (repk.equals("00"))
//if (repk=="00")
{
fld_group="HEADER";
}//else if (repk=="99")
else if (repk.equals("99"))
{
d_penalty=0;
d_amount=0;
fld_group="TRAILER";
String t_initial = line.substring(0,8);
String t_penalty_r = "0000000000";
d_penalty = Long.parseLong(line.substring(8,18).trim());
d_amount = Long.parseLong(line.substring(18,line.length()).trim());
long tot_amount_t = d_amount+d_penalty;
String t_tot_amount = eldb.selectData("SELECT lpad('"+tot_amount_t+"',10,'0') from dual");
line = t_initial+t_penalty_r+t_tot_amount;
}else {
fld_group="DETAIL";
String d_initial = line.substring(0,57);
String d_final = line.substring(71,line.length());
d_penalty = Long.parseLong(line.substring(57,64).trim());
String d_penalty_r = "0000000";
d_amount = Long.parseLong(line.substring(64,71).trim());
long tot_amount = d_amount+d_penalty;
String d_tot_amount = eldb.selectData("SELECT lpad('"+tot_amount+"',7,'0') from dual");
line = d_initial+d_penalty_r+d_tot_amount+d_final;
}

ppsmt.setString(1, file_refno);
ppsmt.setInt(2, kn);
ppsmt.setString(3, line);
ppsmt.setString(4, file_refno.substring(0,10));
ppsmt.setString(5, fld_group);
ppsmt.setInt(6, kn2);
ppsmt.setLong(7, d_penalty);
ppsmt.setLong(8, d_amount);
ppsmt.executeUpdate();
conn.setAutoCommit(false);
System.out.println("im here na @ 3");
}
}
ppsmt.close();
br.close();
isr.close();
ipt.close();

}

Hope anyone can help me or just give me a head start.
Thanks.
 
David Newton
Author
Rancher
Posts: 12617
IntelliJ IDE Ruby
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Please UseCodeTags.

Have you profiled the code at all to see where the code is taking time? Even System.currentTimeMillis() might be enough to help. Obviously it doesn't take "hours" to upload a 4M file, and even complicated SQL processing for a file that small should be a matter of minutes (depending, of course). We have processes here that process files of a million lines or more with fairly complicated SQL on the back end that execute in a matter of tens of minutes.

That aside, I'd sure recommend uploading the file in its entirety and doing to processing "off-line".
 
Deepak Bala
Bartender
Posts: 6663
5
Firefox Browser Linux MyEclipse IDE
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
So each line in the 4MB file contains a HEADER a TRAILER or a DETAIL and you are parsing each line and making a SQL call every time ? That can end up being costly.

That aside, I'd sure recommend uploading the file in its entirety and doing to processing "off-line".


That would be the approach to follow. Write the file to a disk and allow a 'batch' to process its contents. That way the response time for the user can be kept to a minimum.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!