• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Jeanne Boyarsky
  • Devaka Cooray
  • Paul Clapham
Sheriffs:
  • Tim Cooke
  • Knute Snortum
  • Bear Bibeault
Saloon Keepers:
  • Ron McLeod
  • Tim Moores
  • Stephan van Hulst
  • Piet Souris
  • Ganesh Patekar
Bartenders:
  • Frits Walraven
  • Carey Brown
  • Tim Holloway

Partioning with String based key

 
Ranch Hand
Posts: 196
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

I am trying to implement parallel processing for Spring Batch. Basically I have a job that reads from a DB and processes that data. The challenge for me now is to partition that data into different sets so that each thread has a separate partition to work on. Following is some code I found to create the partition...



The problem is that this assumes the column to partition is an Integer, but in my case it is a String (alpha numeric). The values in my column are still unique, they are codes like "AV21" or "BC33" but the above code would not work for it.

Any suggestions on how to re write this to partition for String key?

BTW, I posted this in the Java General forum and not Spring forum because it is not really a Spring question. It is really a plain Java question.

Thanks in advance for any help.
 
Bartender
Posts: 10759
68
Hibernate Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Jehan Jaleel wrote:The problem is that this assumes the column to partition is an Integer, but in my case it is a String (alpha numeric). The values in my column are still unique, they are codes like "AV21" or "BC33" but the above code would not work for it.


I'm still not sure exactly what it is you need; an alphanumeric string can be converted to a number simply by encoding it as base 36. The only problem is that a long can only hold 12 such digits.

Any suggestions on how to re write this to partition for String key?


Well, the normal way would be to take the total number of objects, divide by however many threads you think you need, and get the keys for each breakpoint, but maybe I'm missing something?

]BTW, I posted this in the Java General forum and not Spring forum because it is not really a Spring question. It is really a plain Java question.


Thanks for pointing that out, but I'm not sure that this actually IS a Java question. Seems to me that what you want to know is how to partition a set of values that you want to process. All the database stuff is just hooey.

Be happy to help more if/when you can provide a bit more info - and by that I mean info about the data and how you want to split it; not what you've implemented (presumably, in error) in Java.

Winston

PS: PLEASE take out that line connect at the end of the 'min' defintiion. It isn't necessary, and it drives the window software here mad because it makes the line so long.
 
Jehan Jaleel
Ranch Hand
Posts: 196
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for the response Winston. I guess what I was really asking is how to convert an alphanumeric string to an int (or long) and you answered this question that to use base 36 encoding. Now in my code I am converting to long where needed and then converting back to String...

 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!