Stu Thompson

OK, the answer is unofficially "yes". A vendor with access to a 64-bit system (HW, OS and JVM) ran my test program with success. HP/Intel, RHEL & Sun Java 5.

I suspect that this could vary from system to system though because of the OS-dependancy and implementation specific nature of MappedByteBuffers.

Stu

Hi all,

In the course of performance testing and optimization of my large-file, I/O intensive application, it has become apparent that it could benefit from switching the stream-based input to a NIO MappedByteBuffer approach. The micro-benchmarks look good (5x speed improvement on read operations) and I am working on a production implementation and real-world benchmarks.

One thing that has come up is that there is a fixed number of bytes that can be mapped. Once that number is reached, OS-level IOExceptions are raised. ("Cannot allocate memory" on OSX, something similar on XP.) That number seems to fit this equation:
number of bytes than can be mapped < (2^32 - physical ram - virtual memory)
This makes sense to me, although I have not read anything on MappedByteBuffers that says as much. Unfortunately for me, that number is too small--if I were to put it into production, problems would arise at some point.

The question: On a 64-bit system (64-bit CPU and OS), would the total bytes mapped with MappedByteBuffer increase also? I don't have access to a 64-bit system and cannot test. But I am lucky enough to be able to specify the hardware requirements of my application.

If someone here has a 64-bit CPU running a 64-bit OS, and the curiosity, could you run my test code on your system to see what happens? It should not take long. Just use a large file, and set the 'factor' variable to something wherein file.length() * factor > 4GB. Please include your JVM details as MBBs are vendor specific with regards to MBB implementation. I am most keen to learn about Linux on Intel/AMD.

And, if anyone else has any thing educational to add, please do! There seems to be scant information about MappedByteBuffers on the net.

Cheers,

Stu

private static void nioMbbMultiMap() throws IOException { int factor = 23; File file = new File("/path/to/largeFile.bin"); //file size * factor = total mapped bytes System.out.println("File size: " + file.length()); System.out.println("# of MBBs: " + factor); System.out.println("Total mapped bytes: " + file.length() * factor); FileChannel[] fc = new FileChannel[factor]; for (int i = 0; i < factor; i++) fc[i] = new FileInputStream(file).getChannel(); MappedByteBuffer[] mbb = new MappedByteBuffer[factor]; for (int i = 0; i < factor; i++) { System.out.println("MMB #" + i); mbb[i] = fc[i].map(FileChannel.MapMode.READ_ONLY,0, file.length()); //if there is a problem, it will blow up on the above line } for (int i = 0; i < factor; i++) fc[i].close(); }

I don't think there is a configuration to help you out, but you could write a simple servlet, filter or whatever that checks out the http referral address and responds with (or redirects to) a domain specific favicon file.

Stu

Oh, thank you so much for this! I'd been mucking about with createElementNS (and createAttributeNS()) in all the wrong ways...getting excpetions and what not.

It seems to me that working with DOM and name spaces is a bit disconnected and less 'direct' that I was expecting. E.g.: setting the document name space qualifier with rootEle.setAttribute("xmlns:myNameSpace", "http://www.mydomain.com/dtds/mydtd.dtd") rather than some sort of .addNamespace(URI, name) approach.

Again, thanks a zillion!

Stu

Hi all,

I am struggling with how to implement xml document creation code that has name spaces on select nodes and attributes. What I want to create:

<?xml version="1.0" ?> <root xmlns:myNameSpace="http://www.mydomain.com/dtds/mydtd.dtd"> <alpha> <beta>some text here</beta> <myNameSpace:charlie>some other text there</myNameSpace:charlie> </alpha> <root>

I am working with Xerces 2 and xml-apis, which I would think is capable of such a task. Now, after hours of experimentation and searching the web, I am loosing faith.

Does anyone have some example code for how I can do such a thing? I'd be willing to use other APIs or xml implementations or whatever...anything other than writing raw strings. :p

Perplexed beyond belief,

Stu
[ June 15, 2007: Message edited by: Stu Thompson ]

Hi all!

It�s been about a year now that I joined Java Ranch, and an educational one it has been. I�d just like to thank you all senior ranchers for creating, running, improving, moderating and contributing to the site. My knowledge of Java beforehand was academic (a course in Uni) and managerial (overseeing a team of Java coders.) Now I am in the thick of it and loving every moment!

Fantastic stuff: forums, book reviews, Rules Roundup, FAQs, etc. A very well rounded and relevant site.

A special thanks to William Brogden. Prior to last fall, I has always considered myself �a high-level application guy.� With his initial introduction and encouragement to some servlet stuff (rememeber "read-n-spew" ), I have been jumping much deeper into Java than I ever expected. Also, the resulting servlet project has put a few gold stars beside my name on the company register�so much so that I can wear what I want and nobody complains about the late starts I prefer. (Bill, if you ever find yourself in Z�rich, please drop me a line and I�ll buy you a sarsaparilla!)

Anyway, that�s enough smoke blown your collective way. Thanks again and keep up the great work.

Stu
[ June 04, 2007: Message edited by: Stu Thompson ]

Hi all,

As I understand it, MappedByteBuffers use the OS's native paging system (if available). This is supposed to be a key reason why MappedByteBuffers perform better than BufferedInputStreams.

My question is: what happens if the file is physically located on another machine and is accessed via a Windows share or NFS?

(Context: server-side, read-only, sequential reads, multiple access, large files.)

tia,

Stu

Also, any opinions on glossy vs. non-glossy displays? As far as displays go I think I'm gonna need one of those glare hood thingies for outdoor work.

I am pleased with my glossy screen MacBook. The gloss seems to make text and images nice and crisp. BUT, it reflects any light sources (lamps, windows, etc.) in a near mirror like fashion. I find myself positioning my MacBook just so in the office...might not be best for working outdoors.

Stu

Originally posted by Bill Compton:
Hmmm... uname -a reports:
... sun4v sparc SUNW,Sun-Fire-T200
Does that tell us what processor it is?

I think this is the problem!

Wikipedia has a decent breakdown of Sun Fire servers...and from The Register, it seems that T200 was the initial name for what is now the T2000.

The Sun Fire T2000 has an UltraSPARC T-series CPU. (This chip is also known as Niagra and is notable for having multiple cores running multiple threads.)

If you google 'niagra fpu' you will see many results regarding floating point performance issues.
[ April 11, 2007: Message edited by: Stu Thompson ]

Can we see the Discount.java, please?

If Discount really is an interface, then you cannot actually instance it. Yes, you can instance FixedDiscount which implements Discount, but that is different.

I have never used an interface with hibernate and am suspecting that your hibernate mapping does not accurately describe your interface and two implementing classes. Therefore, I think hibernate is trying to instance a Discount class (java.lang.NoClassDefFoundError: omwa/model/order/discount/Discount) but cannot because it is actually an Interface...and then blows up.


If Discount is actually a proper super class of FixedDiscount, then I don't know what to suggest. I have a very similar arrangement...but my mapping is simpler: E.g.: no proxy attribute, abstract atrribute, etc.

Stu

Hi,

Exactly what CPU are you running Solaris on?

I found this thread very facinating and when out on 'the google' and found this post: http://forum.java.sun.com/thread.jspa?messageID=9384302&tstart=0#9384302

In a nut shell, the UltraSPARC T1 processor's FPU does *not* include a sqrt instruction. From the link: "computing a sqrt would take 25000 cycles...When executing a sqrt the programs traps into the kernel to emulate the instruction."

25000 cycles seems like alot and has that a couple orders of magnatude scale your problem has. Maybe this is the root (haha) of the problem?

Stu

If discount is an interface, and new FixedDiscount() works, then the error message seems to be saying (and I might have this wrong) that your code is trying to instance an interface and not the class itself...hence the java.lang.NoClassDefFoundError: omwa/model/order/discount/Discount

It might help if you post the hibernate mapping file.

Stu

Has anybody here switched from Hibernate to iBatis? Was it painful? Could you please share your experience?

Thanks in advance,

Stu

Originally posted by Peter John Jones:
...if there is alimit to how many products Tomcat can store providing we have unlimited drive space. (A Product could be anything, word document, jpeg, mp3 etc). We're worried if 10,000+ products are stored could Tomcat handle this and if so how would performance be drastically affected.

If I understand you correctly, which I am not sure if I do because the question seems unusual, you are asking how many files a tomcat server could potentially serve up. To that specific question, the answer is "practically unlimited". Tomcat does not "pre-load" any static files for serving up, it does not scan the non application files (files in the webapp but not in WEB-INF) or any such thing. In fact, I cannot think of any web server where this would not be true.


Beyond that, like if you have logic behind each request, well that is a matter of application design and implementation...and the the limiting factor is you

I came across Filip Hanik's recent blog post on Tomcat scaling vs. Glassfish and Jetty a few minutes ago and thought of this thread. Hope this helps.

Stu
[ March 22, 2007: Message edited by: Stu Thompson ]