Multi-Core Vs. Single-Core (For the serious Java Archs)

[Raj Chaud]

Sup folks. I'm new to this forum, anyway, I thought I'd start a thread. Actually, I think I'm going to try and come here often, looks like interesting stuff.

Anyway, who wants to discuss multi-core versus single core??? This is the real topic of discussion now in J2EE architect world.

Take a scenario:

Exact same code + 2 CPU single core = ???
Exact same code + 1 CPU 8 core 4 thread/core = ???

What is the answer??? Where will your application run faster on Java 1.5? Remember Java 1.5, not Java 1.6 (where we know the language is just mucho faster anyways)....................

The answer? Not really much difference! Once again, the same Java application, running the same code won't prove to run much faster on the 8-core 32 virtual CPU scenario above. The answer is, the nature of the application is not coded to take advantage of the extra juice. The small gain is from the fact that the Java language is running a tad bit faster in 8-core environment.



Anyone care to comment??? I would love to hear anyone else's views on this topic or your thoughts, or the results of any experiences you may have had.

thanks

[Nicholas Jordan]

Well it should, dispatching to multiple cores, busses or whatever nomenclature may be used is profoundly dependent on implemtation of parallelizing compiler. In fact, I wrote my first Java program intending to take advantage of scalability to Solaris ( as a base platform for prototyping ) believing that migration across commercial domains would resolve to writing test harness (s) to detect conformant implementation on candidate customers. Here's the actual code, little changed from some bloated discussions at JR nearly a year and a half ago:

try
{
    java.lang.String osName = System.getProperty("os.name");
    // Whether we are working around Solaris.
    final boolean solarisWorkaround = (osName != null &&
                                       osName.indexOf("Solaris") != -1);
    /* ==== Loop counter. === */
    int loopIndex = 0;
    //
    /* ==== Thread management. === */
    final java.lang.Thread currentlyExecutingThread 
              = 
              java.lang.Thread.currentThread();
    final java.lang.Integer betweenThreadLaunches 
               = 
               new  java.lang.Integer(java.lang.Thread.NORM_PRIORITY - 1);
    //
    /**
     * on Unice, Max File Handles == 60
     * on M$ #define FOPEN_MAX 20
     */
    final java.lang.Integer MAX_THREADS 
                         = new java.lang.Integer(solarisWorkaround?60:20);
    final java.lang.Integer MAX_OPEN_FILE_HANDLES
                         = new java.lang.Integer(MAX_THREADS.intValue() - 4);
    final java.lang.Integer BUFFER_SIZE
                         = new java.lang.Integer(solarisWorkaround?8192:4096);//
// and so on .....

It initially ran close to 1,000 lines because I suffered beginner grandeur and stuffed it to resemble finished size, since cut back to about 500 lines.
[ June 11, 2008: Message edited by: Nicholas Jordan ]

[Raj Chaud]

Nick, this code uses your own thread programming. That makes a difference. I thinking more about older code. Just the plain old code you used to write 3 years ago before the multi-core CPU explosion.

The question is, have you ever seen that run faster on multi-core than on single-core? i.e., with absolutely no changes.

[Nicholas Jordan]

Well why would it? We get a wide range of skills here, just to ask the question suggested to me you would recognize the need to apply the linguistic effectively, that was the first program I wrote in Java.

Why would one NOT code to parallelization?.....

That would be an app that could gain nothing from multi-core, no?

[Peter Johnson]

For a single-threaded Java app, there is no benefit to going with multiple cores. For a multi-threaded app (such as a Java EE application server) there are significant benefits going multi-core.

I have written demo apps that scale linearly up to 32 processors. If I had more cores I could have gone higher. But that was a very artificial app that ran completely independent threads, each of which had a small enough working set that it fit into the processor cache.

However, when the threads interact, or depend on shared or synchronized data, then there is usually a sweet spot. For example, 8 cores might get the best performance, while using either more or fewer cores results in diminished performance (here performance being throughput). The only way to determine the sweet spot for your app is to run it under various configurations and tune it.

[Rusty Shackleford]

"Why would one NOT code to parallelization?....."

Because the program you are writing is not inherently parallel.

Because the extra time it takes to refactor it outweighs any speed benefit.

Because the code you programmed in parallel runs slower on multiple cores.

At least on the desktop, the future is not parallelized programs. In my opinion, the future is the OS running a process on one core, another on the second, etc. In many cases, be it desktop, server or whatever, this will be a more optimal solution.

If a program, no matter what space is runs in, can be made to run in parallel that doesn't mean it is going to run faster. Take an extreme example: a program that has be written so that 80% of the execution time can run in parallel will not double in speed on 2 cores. It won't speed up 4x on cores, etc. In fact, with 4 cores you won't see a 2x performance boost. It all depends on the program and the processor architecture. Do the separate L1 caches need to communicate often? Is L2 shared? Write-through? Write-back?

"Take a scenario:

Exact same code + 2 CPU single core = ???
Exact same code + 1 CPU 8 core 4 thread/core = ???"

There is not enough information to be able to respond in any meaningful way. Java may hide the hardware details from the programmer but that doesn't mean it is irrelevant in terms of performance.
[ June 11, 2008: Message edited by: Rusty Shackleford ]

[Raj Chaud]

Originally posted by Nicholas Jordan:
Well why would it? We get a wide range of skills here, just to ask the question suggested to me you would recognize the need to apply the linguistic effectively, that was the first program I wrote in Java.

Why would one NOT code to parallelization?.....

That would be an app that could gain nothing from multi-core, no?

Nicholas, I don't even understand what you are getting at with that "linguistic" comment, so I am going to ignore it.

Anyway, you're missing the point. The whole idea, or selling points behind JAVA (and I've been doing this since just about they came out with this thing over a decade ago) was platform independence, pure scalability, and oh yeah, platform independence.

the whole idea was to let developers spend time writing effective code that would perform business tasks and support a business, not for us old devs to see how good we could get at it, although, to some extent it has turned out that way. :-))

point being bro, if I have to go about coding too specific for the platform, then why the hell am I still not writing C++? may as well manage the memory, and threading, etc, etc, etc from there YES? No!

That is the whole point. So the question is, if I hadn't written any platform specific code (and anyone that doesn't think that writing code specific to the machine is platform specific = CRAZY) like the normal dev on my team would do 3 years ago, would it speed up if I went to a multi-core machine???

the answer I have found is NO, well, i.e., not much. java does execute a little faster, but the gains are minimum. so, I know why this is, but I was just asking if everyone else had similar experiences...............because, this will set us up in the industry in the next near period to discuss how we can speed up single threaded code from a machine standpoint :-))

[Raj Chaud]

Originally posted by Peter Johnson:
For a single-threaded Java app, there is no benefit to going with multiple cores. For a multi-threaded app (such as a Java EE application server) there are significant benefits going multi-core.

I have written demo apps that scale linearly up to 32 processors. If I had more cores I could have gone higher. But that was a very artificial app that ran completely independent threads, each of which had a small enough working set that it fit into the processor cache.

However, when the threads interact, or depend on shared or synchronized data, then there is usually a sweet spot. For example, 8 cores might get the best performance, while using either more or fewer cores results in diminished performance (here performance being throughput). The only way to determine the sweet spot for your app is to run it under various configurations and tune it.

"For a single-threaded Java app, there is no benefit to going with multiple cores."--not entirely true bro, you can still collect garbage in parallel, whether you're in an app server or not.

the bigger question is not about load. Let's say you have an app sitting on an app server. if you didn't do any thread programming, which you probably shouldn't have to, then most of your requests execution will carry on through a single thread. So, even though your app server has multiple threads available internally, your app is still executing in a single thread making it a single threaded application in essence.

so, what do we do about this? if I use multi-core, if getting something out of the DB and serving it up took me 1 second before, will it be faster now? especially, under heavy load?

the answer I've found is YES and NO. Yes, it is faster, but many times faster??? not even.

However, I do agree with you, overall you can execute faster in multi-core, if your code per request is multi-threaded. However, please remember if you are in an app server, don't start generating your own threads, try and use the app server's own thread pool first if you can.

[Raj Chaud]

Originally posted by Rusty Shackleford:
"Why would one NOT code to parallelization?....."

Because the program you are writing is not inherently parallel. - True Dat!

Because the extra time it takes to refactor it outweighs any speed benefit. - True Dat!

Because the code you programmed in parallel runs slower on multiple cores.-Sometimes, but mostly if you have contention bottlenecks

At least on the desktop, the future is not parallelized programs. In my opinion, the future is the OS running a process on one core, another on the second, etc. In many cases, be it desktop, server or whatever, this will be a more optimal solution.

If a program, no matter what space is runs in, can be made to run in parallel that doesn't mean it is going to run faster. Take an extreme example: a program that has be written so that 80% of the execution time can run in parallel will not double in speed on 2 cores. It won't speed up 4x on cores, etc. In fact, with 4 cores you won't see a 2x performance boost. It all depends on the program and the processor architecture. Do the separate L1 caches need to communicate often? Is L2 shared? Write-through? Write-back?

"Take a scenario:

Exact same code + 2 CPU single core = ???
Exact same code + 1 CPU 8 core 4 thread/core = ???"

There is not enough information to be able to respond in any meaningful way. Java may hide the hardware details from the programmer but that doesn't mean it is irrelevant in terms of performance.


Dude, I agree, but "There is not enough information to be able to respond in any meaningful way"--that is the whole point...............

once again, JAVA, we are supposed to be the platform independent, highly scalable, platform performant model..................right? that was the whole point.

so what I am asking is vague no doubt, but I am not asking for evidence or numbers here, just the discussion. If the code doesn't explicitly thread in someway, if I go from a single core to a dual core, will I go faster? my answer again, is Yes, but not by much.......
[ June 11, 2008: Message edited by: Rusty Shackleford ]

[Paul Clapham]

"El Gato Malo", please check your private messages regarding an important administrative matter.

[Rusty Shackleford]

For a single-threaded Java app, there is no benefit to going with multiple cores."--not entirely true bro, you can still collect garbage in parallel, whether you're in an app server or not.

Generally speaking, if the GC needs to run so much that it is seriously impacting performance, a code review might be in order.

we are supposed to be the platform independent, highly scalable, platform performant model..................right? that was the whole point.

IMO, OS independence is closer to the truth. You simply can't expect a given program to perform equally well(relatively speaking) on all the hardware that these OS's run on. "Write once, run anywhere" does not imply anything about performance across the different architectures.

What I mean is a desktop or a general purpose server app couldn't possibly be tweaked to run at maximum performance for all possible processor architectures. This is not necessarily the case in the enterprise sector of development.

I think you could consider this a cost of the platform independence nature and relative ease of development that Java provides. Just like improving performance in a program is often at the cost of more memory usage.
[ June 11, 2008: Message edited by: Rusty Shackleford ]

[Pat Farrell]

Multi-core systems (or even old style dual CPU systems) need tasks to be developed for parallel work. If you have sequential code, you want the fastest clocked single CPU. A dual or quad does nothing for you.

Java can use threads. If the smart programmer makes good use of threading on a problem that can be cleanly made parallel, it can have significant improvements. But there are lots of domain problems that don't tend to be obviously split and processed in parallel. And doing it right is harder than most folks think.

Some applications are trivial to split. Consider a big set of JSP/servlets handling lots of users. The servlet engine will do the right thing.

I actually think that Java's threads are too hard to use, and to really use it, we need another language that has a higher level of abstraction. YMMV.

[William Brogden]

A dual or quad does nothing for you.

That assumes the operating system has no other tasks.

Bill

[Nicholas Jordan]

[Rusty Shackleford:]Because the program you are writing is not inherently parallel.  
Well since OP sought to open discussion, I counter that sales will write mono-code path in parallel, invoking sophisticated compiler optimizations ~ then do bulky, tiered ops in squeak - the micro language. This is sorta sophisticated nuance splitting for engineers, but I am sure we see it too often and I have a degreed risk-analysis engineer so I actually seek any feedback. I have better things to do with my time than drivel on masters.

[Rusty Shackleford:]Because the extra time it takes to refactor it outweighs any speed benefit.  
In such arena, the J2EE Architect should deal with the client, they have soft-skills ~ I do not.

[Rusty Shackleford:]Because the code you programmed in parallel runs slower on multiple cores.  
Ditto, that's for the Architect.

[Rusty Shackleford:]At least on the desktop, the future is not parallelized programs. In my opinion, the future is the OS running a process on one core, another on the second, etc. In many cases, be it desktop, server or whatever, this will be a more optimal solution.  
Agreed, largely. Crossing process boundaries is costly in the time domain, selection of supervisor program such as operational allocation of i/o resources may not require dedicated processor architecture and in fact preliminary concepting on asymetric, transfer function of 1/(scale * processorCount + 1) to determine allocation of supervisor circuitry from application circuitry may yield OP's goals in our work.

[Rusty Shackleford:]If a program, no matter what space is runs in, can be made to run in parallel that doesn't mean it is going to run faster. Take an extreme example: a program that has be written so that 80% of the execution time can run in parallel will not double in speed on 2 cores. It won't speed up 4x on cores, etc. In fact, with 4 cores you won't see a 2x performance boost. It all depends on the program and the processor architecture. Do the separate L1 caches need to communicate often? Is L2 shared? Write-through? Write-back?  
Cache consistency is not likely to be achieved effectively in client code presented in routine business. ( ?... speculative look ahead ) In my opinion, some artificial intelligence could be put on BUS_n to do reliability in large-sclae socialization structures that has not yielded to cryptographic efforts. [ the password on the monitor problem ]

[Rusty Shackleford:]There is not enough information to be able to respond in any meaningful way. Java may hide the hardware details from the programmer but that doesn't mean it is irrelevant in terms of performance.  
A classic case of Engineering School being packed with attentive, focused minds sweating on pencil and paper dreading time consuming calculations has been replaced ( speculative ) with hoards of role-playing minds fascinated to The Glass Eyed Monster. { CRT }

[Raj Chaud:]Nicholas, I don't even understand what you are getting at with that "linguistic" comment, so I am going to ignore it.   Linguistic is a generalized terminology for human-readable code such as Java, C/C++, Perl, Lisp, ADA and any of the written codes vis-a-vis some speculative Point & Program tool such as IDE's could potentially become, if not already.

[Raj Chaud:]Anyway, you're missing the point. The whole idea, or selling points behind JAVA (and I've been doing this since just about they came out with this thing over a decade ago) was platform independence, pure scalability, and oh yeah, platform independence.   I missed nothing in your post, I calcualted my comments EXACTLY to pull in the heavyweights we see posting. Now that I have some twenty-thousand-pound-elephants, watch me make them dance!

[Raj Chaud:]the whole idea was to let developers spend time writing effective code that would perform business tasks and support a business, not for us old devs to see how good we could get at it, although, to some extent it has turned out that way. )   What Java is becoming in fact is an effective training platform for CS students. That was ( opening the door to open development ) how the mighty M$ publishing empire came to be on the front of software innovation. ( spare me, folks ) Then we have a development concept ( Java ) by what may be modeled as a competition player, not-dissimilar, evoking very effective discussions such as we have here. I doubt we will rule, such as experience shows, but for those of us who want to keep up with issues in J2EE so that we know what will be coming in the door, then we have a cross-platform linguistic we are skilled in and thus may not be wrapped is silk but nevertheless make our work here productive.

[Raj Chaud:]point being bro,   Please do not use variant spellings, it drags beginners in where Pro-Bro's must hammer the 304 Stainless.

[Raj Chaud:]if I have to go about coding too specific for the platform, then why the hell am I still not writing C++? may as well manage the memory, and threading, etc, etc, etc from there YES? No!   Proposals, anyone?.... (snicker, snicker)

[Raj Chaud:]That is the whole point. So the question is, if I hadn't written any platform specific code (and anyone that doesn't think that writing code specific to the machine is platform specific = CRAZY) like the normal dev on my team would do 3 years ago, would it speed up if I went to a multi-core machine.Why do you want to speed it up?

[Raj Chaud:]the answer I have found is NO, well, i.e., not much. java does execute a little faster, but the gains are minimum. so, I know why this is, but I was just asking if everyone else had similar experiences...............because, this will set us up in the industry in the next near period to discuss how we can speed up single threaded code from a machine standpoint.We speed it up by allocating some un-used processor power to thwarting weekend ruiners.

[Pat Farrell]

Originally posted by William Brogden:
That assumes the operating system has no other tasks.

And usually a safe assumption in this context.

But the OP's question is too general to answer with more precision one way or the other

[William Brogden]

And usually a safe assumption in this context.

I dont see that we have enough information to assume that. Operating systems always seem to be doing stuff behind my back.

Bill