Win a copy of Programmer's Guide to Java SE 8 Oracle Certified Associate (OCA) this week in the OCAJP forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Batch processing

 
Chris Montgomery
Ranch Hand
Posts: 141
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I need to a load a file which consists of transactional data into a database every night. This file may contain 100s of 1000s of records.

Is there an existing framework which has been written to do just this? If not, is there an "accepted" approach to batch process �large� files in java?

I�ve never written a batch job before. My initial thought is to write some sort of multi-threaded job where each thread takes a section of the file and goes back for more until the entire file is done. It seems simple enough, but if a framework/approach is already in place, which plans for things I haven�t thought of yet, I�d rather go that route 

Thanks!
 
Joe Ess
Bartender
Posts: 9312
10
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Batch processing by definition is non-interactive, meaing there's no user waiting for something to happen. Since there's nobody watching, there's no need to get fancy with the processing. Read a record out of the file, set the values in the DB, do it until the file's done. If you want to get fancy, have the process send an email upon completion or error.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic