• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Spring Batch to process 1500000 records in cache memory for each event

 
Ranch Hand
Posts: 814
Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi All,

I am using Spring Batch for following requirement

I have requirement for our Project Asset Liability Mamagement (ALM)

1. One user initiated event will have 1500000 records to process.
2. For every seconds there will be around 1000 events for processing data.
3. We will get input data for processing each record from application.
4. We will read records from cache memory and then will process records and again will write into cache memory. This process is for single record, in same way we have to process 1500000 records.
3. I am using Spring Batch to do above job.

Can anybody give suggestion or comments to achive above job processing in less time with optimum performance.

 
Ninad Kulkarni
Ranch Hand
Posts: 814
Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi All,

Please give comments and views on above.
 
ranger
Posts: 17347
11
Mac IntelliJ IDE Spring
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Not really much to go on for anyone to answer. But I have some suggestions.

Or better yet, the docs of Spring Batch has those suggestions

http://static.springsource.org/spring-batch/reference/html/scalability.html

Remote Chunking and Partitioning is the basic answer.

Mark
 
Ninad Kulkarni
Ranch Hand
Posts: 814
Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Mark for your views and suggestion we have used same
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic