Unicode convert problem with Jboss in Linux

[isuru chamarasinghe]

Hello,

I have faced with an issue where UTF-8 characters are not correctly formatted ONLY in the Jboss (jboss-5.1.0.GA-3) instance running on Linux.

For an instance: BORÅS is converted to BOR?S at Linux jboss instance.

When I copy and configure the same jboss instance to run at Windows (SP3) it works perfectly.

Also I have change the default setting in Linux by including JAVA_OPTS=-Dfile.encoding=UTF-8 in .bashrc and run.sh files.

Any suggestions or workarounds ?

[Rene Larsen]

Are your JSPs, HTMLs, property files etc. saved with the UTF-8 encoding ??

[isuru chamarasinghe]

Hello Larsen,

Thank you for the prompt reply.

I have exposed a jsp to upload a xml file using the post method. I will be obtaining the file as a FormFile using struts in the Action class as follwos:

FormFile file = theForm.getFile();

Secondly, I retrieve the file data as byte array

byte[] buf = file.getFileData();

When I view the result of above buf variable at runtime it is clear that the unicode has been converted to a non recognized character.

[Rene Larsen]

Try put URIEncoding="UTF-8" in the HTTP connector - in the file JBOSS/server/default/deploy/jbossweb.sar/server.xml

Like this:

<Connector protocol="HTTP/1.1" port="8080" address="${jboss.bind.address}" 
           connectionTimeout="20000" redirectPort="8443" URIEncoding="UTF-8" />

[isuru chamarasinghe]

I include the

URIEncoding="UTF-8"

attribute to HTTP Connector tag in jbossweb.sar/server.xml and restart the instance. No luck. same result.

[Rene Larsen]

And you have tried all these steps too ??

1) struts-config:

<controller contentType="text/html;charset=UTF-8" nocache="true" processorClass="org.apache.struts.action.RequestProcessor" />

2) included a jsp page directive:

<%@ page contentType="text/html;charset=UTF-8" pageEncoding="utf-8" %>

3) included a meta tag within the html page itself:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8">

What happens IF the uploaded file has the right encoding - UTF-8 ?? and not an unknown randomly type.

[isuru chamarasinghe]

Hello Larsen,

I have tired out all those but same result thank you very much for the guidance. Now I have identified the problem is on the server side and not in the view.

I realize that by trying to save the byte array obtain from the File into an xml file in jboss tmp folder and it contains the correct Unicode.

In my scenario the uploaded file is straight away sent to a message queue to be processed by an MDB. So the flow is as follow:

Upload the XML ---> Obtain the byte[] from the file ---> put it on a JMS queue ---> picked by a MDB to process

When the posted Message is cast to a TextMessage and process, the Unicode is been incorrectly converted.

Is there is any specific configuration I have to do in the JMS queue in order to convert it to the correct Unicode?

[Rene Larsen]

On google I found this answer: http://stackoverflow.com/questions/2973576/encoding-a-jms-textmessage

Maybe it also works on JBoss (it should work there too)

[isuru chamarasinghe]

No Larsen,

It doesn't work with Jboss MQ.

I bedbug the Message Listener and found out that JBossTextMessage.getText() contains the file data with incorrect Unicode characters.

Any way of specifying the type of the Message before putting it to the Jboss MQ?

[Jaikiran Pai]

JBoss AS-5 does not use JBoss MQ. It uses JBoss Messaging. Are you sure you are looking at the right classes?

[isuru chamarasinghe]

Yes it is Jboss Messaging as Jaikiran said but do you think this will have an effect on this Unicode issue ?

[isuru chamarasinghe]

Finally I was able to find a solution, BUT the solution is still a blackbox. If anyone have the answer to WHY it has failed/successful please update the thread. Till it is solved I will let the thread open.

Solution at a glance :
1. Captured the file contents as a byte arry and wrote it to a xml file in jboss tmp folder using FileOutputStream
2. When posting to the jboss Message queue, I used the explicitly wrote xml file (1st step) using a FileInputStream as a byte array and pass it as the Message body.

Code example:

View: JSP page with a FormFile

Controller Class :UploadAction.java

public ActionForward execute(ActionMapping mapping, ActionForm form, HttpServletRequest request, HttpServletResponse response){
   ...........
   
   writeInitFile(theForm.getFile().getFileData()); // Obtain the uploaded file

   Message msg = messageHelper.createMessage( readInitFile() ); // messageHelper is a customized factory method to create Message objects. Passing the newly    
   wrote file's byte array.

   messageHelper.sendMsg(msg); // posting in the queue

   ...........
}

private void writeInitFile(byte[] fileData) throws Exception{
   
   File someFile = new File("/jboss-5.1.0.GA-3/test/server/default/tmp/UploadTmp.xml");  // Write the uploaded file into a temporary file in jboss/tmp folder
   FileOutputStream fos = new FileOutputStream(someFile);

   fos.write( fileData );

   fos.flush();
   fos.close();    	
}

private byte[]  readInitFile() throws Exception{

   StringBuilder buyteArray=new StringBuilder();

   File someFile = new File("/jboss-5.1.0.GA-3/test/server/default/tmp/UploadTmp.xml");	 // Read the Newly created file in jboss/tmp folder

   FileInputStream fstream = new FileInputStream(someFile);

   int ch;
   while( (ch = fstream.read()) != -1){
   		buyteArray.append((char)ch);
   }
   fstream.close();

   return buyteArray.toString().getBytes();   // return the byte []
}

Foot Note: I think it is something to do with the Linux/Windows default file saving type. eg: Windows default : ANSI.