Win a copy of Functional Reactive Programming this week in the Other Languages forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Build a file in Memory

 
Richard Butterwood
Ranch Hand
Posts: 45
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
An XML file is returned to me via a http connection. Currently I write the XML file out to a temporary file on the OS. After the file is completed I return the temporary filename which another uses to manipulate the XML file.

Is it possible to build the file in memory and just return the file object?

If so, can you supply example code?

Thanks!
 
Scott Selikoff
author
Saloon Keeper
Posts: 4028
18
Eclipse IDE Flex Google Web Toolkit
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sure it is, if you are receiving information using Http request, build the file as a stream then output the file to the user using Http response. In order to transmit just the file you have to change the content type of the stream to direct the browser to download this file.
[ November 17, 2005: Message edited by: Scott Selikoff ]
 
Richard Butterwood
Ranch Hand
Posts: 45
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Why do I need to use http respone? At this point I have finished with http and have the XML file in a nice little file object.

What I want is to close the input and output stream and just return the file object.

Can this be done?
 
Ernest Friedman-Hill
author and iconoclast
Marshal
Pie
Posts: 24212
35
Chrome Eclipse IDE Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yes -- well, sort of. You can't return a java.io.File object without creating an actual file on disk, because that's what a File is. But you can return, for example, a Reader from which the other code can read the XML. If you've got the downloaded XML in a String, then just directly construct a java.io.StringReader from it, and return that. Otherwise, the class ByteArrayInputStream lets you create an InputStream that returns an arbitrary collection of bytes -- use InputStreamReader to turn that into a Reader if you need it.

Does that answer your question?
 
Richard Butterwood
Ranch Hand
Posts: 45
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Ernest,

Thank you for the reply and yes it does answer my question.

Is there a size limit in a string buffer or string reader? Some of these XML files are going to be very big.
 
Ernest Friedman-Hill
author and iconoclast
Marshal
Pie
Posts: 24212
35
Chrome Eclipse IDE Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
No limit other than available memory.
 
Paul Clapham
Sheriff
Posts: 21416
33
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The size limit for a string or an array is 2^31-1 which is about 2 billion. However the practical limit is the amount of memory you can have. It would be preferable to store the XML as a byte array rather than as a String, since unless your XML is mostly Asian characters the bytes will only take up half as much room as the chars.

But you have a process that just reads the XML from the HTTP connection and stores the data, then passes a reference to the data to some other process that parses the data? Why? Why not just let the second process read its data directly from the HTTP connection?
 
Richard Butterwood
Ranch Hand
Posts: 45
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Here is the code that I use to build the XML file:

OutputStream outStream = new FileOutputStream( XMLFile );
InputStream inStream = connection.getInputStream();
int c;
while ((c = inStream.read()) != -1)
outStream.write(c);
inStream.close();
outStream.close();

Are you saying that I should return "connection.getInputStream()" to the calling program?

I am using XML Beans to parse the XML. I suspect that XML Beans requires a physical file to use.
 
Clifford Garvis
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Do you want it in memory to speed up performance? If you want it in memory you could pull it into a MappedByteBuffer
 
Richard Butterwood
Ranch Hand
Posts: 45
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yes, I am going down this road for performace reasons. It seems a waste to dump it out to a file, just to read it back in again.

Do you think performance would be a hit doing it like this?
 
Clifford Garvis
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It all depends on how much work you are really doing... unless you are reading in very large files and doing alot of I/O and modifications I don't think you will see much difference. Doing it in memory is faster but you probably have alot more space available on your storage device then you do in memory.
 
Paul Clapham
Sheriff
Posts: 21416
33
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Are you saying that I should return "connection.getInputStream()" to the calling program?
Exactly.
I am using XML Beans to parse the XML. I suspect that XML Beans requires a physical file to use.
This is the XML Beans from Apache? Given that it's written by Apache I expected that there would be overloaded methods that allowed you to read XML from a file, a reader, an input stream, and so on. Any competently written XML software should have those, restricting you to input from a file would be poor design. And when I looked at the API documentation for the product I saw that the various parse() methods did indeed have those overloaded methods. You can even parse directly from a URL.
 
Jeff Albertson
Ranch Hand
Posts: 1780
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
On XMLBeans: yes, the various parse methods of Factories can take a File,
InputStream, Reader, URL, etc... and conversely, the save methods can
take a File, OutputStream, Writer etc...

You should realize that XMLBean is built on top of DOM, so it is not
promising better speed or memory footprint than DOM. Have you
considered something lighter weight, like SAX, XSLT or StAX?
 
Richard Butterwood
Ranch Hand
Posts: 45
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Some of the XML "Responses" will be very small, but some will be very large.

Are you saying that XML Beans will be too much of overhead to use?

If performance becomes an issue I will investigate XLST.
 
Jeff Albertson
Ranch Hand
Posts: 1780
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Why not write some test cases and see if the performance is satisfactory?
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic