Win a copy of The Java Performance Companion this week in the Performance forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Batch updation failure because of duplicate records

 
Sandeep Swaminathan
Ranch Hand
Posts: 52
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm trying to upload the records from a CSV file into the database. On the process I'm trying to validate those records extracted from the CSV file. Because of performance issue, I thought of using batch update which will return me an array of successfully loaded index. But even if there's a duplication of one record, the entire batch is failing and the records are not inserted into the database. Am I not supposed to use batch update for such cases? Then how do I resolve the performance and cost issue as I dont want to hit the database for each and every insertion of record.

NOTE: I'm not supposed to use LOAD DATA INFILE and use only INSERT INTO command.
 
Ireneusz Kordal
Ranch Hand
Posts: 423
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Load data into temporary table, create index(es) on this temp table (to make query faster),
then execute query to see if duplicates exist and remove duplicates,
then update the table.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic