Saifuddin Merchant

Ranch Hand
+ Follow
since Feb 08, 2009
Saifuddin likes ...
Firefox Browser Java Spring
Cows and Likes
Cows
Total received
0
In last 30 days
0
Total given
0
Likes
Total received
19
Received in last 30 days
0
Total given
108
Given in last 30 days
0
Forums and Threads
Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by Saifuddin Merchant

I personally like http://javabrains.koushik.org/topics/spring
I think it's well organized and goes step by step.
3 years ago
I like the tutorials koushik has come up with - they step by step video format & nicely done = http://javabrains.koushik.org/
4 years ago
if your looking to kick of a "new job instance" (and the word instance is very important) - it doesn't matter what type of executor is being within that job.
P.S. your is a very peculiar requirement. Kicking of a "batch job" for each message that comes seems like a over kill (also might be hard to control concurrency related issues)
4 years ago

Renjith Panikar wrote:One more question.

What about using Spring's 'ThreadPoolTaskExecutor', there we can handle the Queue size?



You could use any Thread Pool Executors - those provided by Spring or those provided by the java.util.concurrent framework. Spring Batch will buy you more than threading - batch re-startability, batch steps, execution tracking etc. Again depends on your use case. If you have no need for any of the "batch features" provided by a framework - plain vanilla java muli threading code (using ThreadPoolExecutor) should work fine.
4 years ago
+1 on using a batch framework (spring batch) to do the plumbing task of multi threading while you focus on the business logic
4 years ago

b. JDBC template is thread safe once instantiated.

You are right. That is one reason i kept the JdbcTemplate creation outside of all threads. But seems like the point given by Jayesh contradicts with this.


The JDBC template is thread safe. As to the underlying driver and how transactions get managed is an other problem. What Jayesh is saying that by opening multiple threads you cannot "commit" all 100,000 transactions. The "commit" will happen for each 10 pieces of 10,000 records. That means if say 9/10 "chunks" are successful - you could end up with 90,000 records in the database. Depending on the business scenario this may or may not be acceptable.

Note that the threads will not be able to share a transaction. Each thread will have tok open it's own connection to db and each connection will have its own transaction

@Jayesh By opening so much connection, is it possible that it can consume most of the connections in the DB2 connection pool?

@Saifuddin What's your thought? if we use a single JdbcTemplate, i think it will take the same amount of time as that of executing single batch at a time.



I would suggest that you try this out and see how it performs. 10 threads should not consume all connections on DB2 pool, but I don't think you will gain a lot in performance (depends on you split method too).
Are you looking at this design purely for a performance reason? Do you really need the gained performance (many batch systems can work with slightly delayed times) - 100,000 is not very big number of records.
4 years ago
Couple of points

a. List<String> splittedRecords = splitRecords(fullRecords); - should be outside the loop. you don't need to run it 10 times.

b. JDBC template is thread safe once instantiated.

Instances of the JdbcTemplate class are threadsafe once configured. This is important because it means that you can configure a single instance of a JdbcTemplate and then safely inject this shared reference into multiple DAOs (or repositories). The JdbcTemplate is stateful, in that it maintains a reference to a DataSource, but this state is not conversational state.



However executing so many database updates might have other problems that you need to consider. What happens when one of those 10 sets fails to update? When do you want to commit the records?
Doing so many inserts in parallel could also cause performance degradation v/s a gain in case you DB has to checks before inserting those records (say auto generated primary key). I'm not an expert on the DB side - so this a wild guess - but you should do some extensive testing.

Would I recommend it? Probably not - unless you have a good reason to spawn multiple threads
4 years ago
I can see a typo - you have spelled read committed as READ_COMMITED while it should have been spelled with a "TT" as READ_COMMITTED
4 years ago
You could do a couple of things ...

a. You could always write a query that can "read" from two different tables. You could map the result from these two queries to an object or a composite object containing two objects you want to map to and finally write it out. Just usual usage - nothing different required

assumption: both you tables are in the same database and can be access via a join query

OR

b. You could use a composite Item reader. Here is an example of how that could be done.
https://github.com/langmi/spring-batch-examples-readers/blob/master/src/main/resources/spring/batch/job/readers/jdbc/jdbc-composite-item-reader-job.xml
4 years ago
Spring certifications are available via VMware- http://mylearn.vmware.com/mgrReg/plan.cfm?plan=31110&ui=www_cert

Basically 3 certification - each one is independent of the other - though if you are starting of - the first one to do would be the Spring Professional
Like Jeanne has pointed out it's a combination of classroom training followed by a certification exam (you have 1 year from end of course to give the exam.)

A really good start place to get all the details related to the exam would be http://springcert.sourceforge.net/

I believe that includes a good section from 1.2 Question Breakdown By Topic
This breakdown is not "official". See Jeanne Boyarsky*'s Spring 3.X Certification Experiences for more details.

* Thought I recognized the name from somewhere
4 years ago

Prerana Verma wrote:@Saifuddin Merchant : By Creating anonymous class for Runnable or Thread.



At the end of the day your still using Runnable or Thread.

So an answer to your question "Is it possible to create...." is No. Whatever you do you will be either extending Thread or implementing Runnable either directly or indirectly.
4 years ago

Ulf Dittmer wrote:Sometimes the best answer to a question is Mu



Learned (learnt) something new and useful
4 years ago

Prerana Verma wrote:Thanku for replying...

It’s just a java beginner’s query. I was trying to implement that but didn’t find any way (Found one by using anonymous class).
Just wanted to know if there is any other way of doing this.



How does an anonymous class allow you to do that?
4 years ago

Dian D Chen wrote:Thank you so much! It works now~
BTW, when I read the tutorial from Apache Maven, it mentioned about setting M2 sys variable. Is that M2 same as this ".m2"?



M2 (M2_HOME) is the system variable that would point to the .m2 directory on disk. It similar to how the JAVA_HOME points to the Java home directory or CLASSPATH points to classes that should be included when java is invoked from the command prompt. Similarly M2_HOME gets used when Maven commands are run from the classpath
4 years ago
+1. I had the same question "What would be a good reason to use Hudson over Jenkins"
We used to use Hudson and have now moved over to Jenkins and as a user I don't see much difference .... pretty similar looking UI. Why use one over the other?
4 years ago