• Post Reply Bookmark Topic Watch Topic
  • New Topic

Convert OutputStream to InputStream!  RSS feed

 
bob connolly
Ranch Hand
Posts: 204
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello!

This post probably looks like it belongs in the XML category group, but it really has more to do with converting an

OutputStream to an InputStream.

I'm trying to intercept an output file "out.xml" and re-route it into another function, instead of writing it out to an

EXTERNAL file!

The reason i'm trying to do this is that the generated XML file is very large, 2 billion recs, and i only need to run the XML

file thru a SAX.parse() - validation routine and create a report showing which records have invalid data, and would like to

avoid creating a huge XML file when it's really not needed!

So basically, i'm trying to convert and OutPutStream back into an InputStream and i've been trying the PipeOutputStream

approach, but am not making very good progress!

Thanks very much for any ideas!
bc

Here is what Oracle is saying to use for LARGE XML SAX processing and my Pipe.... code follows this example:



Here are my attempts to intercept that "out.xml" file and re-route it to the "sp.parse()" function!

 
bob connolly
Ranch Hand
Posts: 204
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I found this in one of the O'Reilly Java in a Nutshell, so i'll be giving this a once over x 5, but if you may happen to have a simpler solution, i'd be glad to hear about it!

Thanks very much!

 
bob connolly
Ranch Hand
Posts: 204
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello!

This is a followup from a previous post on using PipedInputStreams!

I've been trying to use a PipedInputStream approach as a way to validate some VERY LARGE XML datasets and i'm continuing to get
an "OutofMemory error", which is very suprising because the PipedInputStream approach is supposed to Buffer
the data thru memory!
And in these programs i'm also using 2 seperate threads!

Below i have the progams listed, SX4 uses the OraPipe1 to set up the InputStream and
the OraPipe1 calls the OraPipe2 which retrieves the Oracle data and builds the XML

SX4 > OraPipe1 > OraPipe2

--------------------------------------------------------------------------------------------------------------------------
 
bob connolly
Ranch Hand
Posts: 204
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello!

I think i found the problem, the OutofMemory is from the size of the JDBC Resultset, so i'm now using Oracle's SAXSerializer and routing that output to that PipedOutputSteam in the OraPipe2 and it's working fine!

So thanks much for reviewing this problem!

bc
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!