Parsing Chinese Characters by using Xerces

[Ed Tang]

May I know does Xerces support Chinese Characters?

My XML file is encoded in UTF-8 format and including some Chinese characters in some tags. However, when I try to print the characters in char[] data (which are Chinese characters) in the method 'characters', some strange characters are returned '???'. May I know how to get the correct Chinese characters after getting the char[] data?


public void characters(char[] data, int start, int length){
.......
}

p.s. The platform is AS/400.

Thanks a lot.

[Paul Clapham]

Xerces supports all characters which XML supports. And XML certainly supports Chinese characters as you can see by reading the XML Recommendation.

However your question seems misguided to me. You complain about seeing question marks when you print those characters, but then you ask how to get them. It's equally probable that the encoding failure occurs when you try to print the characters which have been correctly got from Xerces.

So you're going to have to explain your process in a bit more detail, not just focusing on Xerces. Especially if you're printing data, which on your platform might well involve some more data conversions.

[Ed Tang]

Here is my code...
===============================================

String schemaFileNameStr = "abc.xsd";
URL schemaUrl = this.getClass().getClassLoader().getResource schemaFileNameStr);
File schemaFile = new File(new URI(schemaUrl.toString()));

InputSource is = new InputSource(new FileInputStream(xmlfile));
is.setEncoding("UTF-8");

XMLReader r = XMLReaderFactory.createXMLReader(parserClass);
TestHandler th = new TestHandler();

r.setFeature(validationFeature,true);
r.setFeature(schemaFeature,true);
r.setProperty(schemaSourceFeature,schemaFile);
r.setErrorHandler(th);
r.setContentHandler(th);
r.parse(is);

===============================================

  public void characters(char[] data, int start, int length)
         throws SAXException {

	
      StringBuffer buf = new StringBuffer(length);
      buf.append(data, start, length);
     
     if (chineseNameTag){
         chineseNameTagValue.append(buf);  //chineseNameTagValue is a StringBuffer
     }

   }

===============================================
When endofElement of 'ChineseName' is detected, write record to AS400 physical file.

AS400 sys = new AS400();
 AS400FileRecordDescription myFD = new AS400FileRecordDescription(sys, 
  "/QSYS.LIB/TEST.LIB/TESTPF.FILE");
RecordFormat[] format = myFD.retrieveRecordFormat();
SequentialFile myFile = new SequentialFile sys, "/QSYS.LIB/TEST.LIB/TESTPF.FILE");
myFile.setRecordFormat(format[0]);
myFile.open(AS400File.WRITE_ONLY, 0, AS400File.COMMIT_LOCK_LEVEL_NONE);

Record writeRec = myFile.getRecordFormat().getNewRecord();
tempS= new String(new String(chineseNameTagValue).getBytes("ISO8859_1"),"CP937");
	
writeRec.setField(1,tempS);
myFile.write(writeRec);
myFile.close();

=========================================

However, the chinese character cannot be written to the file successfully. Hex '3F3F3F' is written instead. When I try to print the value of chineseNameTagVal, '???' is shown in the console....


Could anyone help me? Thanks. I found some hints in the internet, seems need to use the classes of ByteArrayOutputStream & OutputStreamWriter to do some conversion? But I have no idea on how to use these classes....thank you so much!

[Paul Clapham]

tempS= new String(new String(chineseNameTagValue).getBytes("ISO8859_1"),"CP937");

I thought there might be something like this.

You have a string containing Chinese characters. First you convert it to bytes using ISO-8859-1, which is the Western European character set. Since that character set does not include representations for any Chinese characters, it replaces all of them by question marks before converting them to bytes.

So already you have mangled your data beyond recognition. Converting those question-mark bytes back to chars using the CP937 encoding cannot possibly bring back the original data.

I would just get rid of this line entirely. It's the job of the AS400 and SequentialFile objects to convert from chars (in the Java program) to bytes (in the database), not yours. Just make sure that the job where this running and the database tables both have a suitable CCSID.