Sevlet encoding for non UTF-8 characters

[rajkumar jayavel]

Hi whenever i ran my following code its working in the standalone application
While i am trying in the servlet its showing invalid(?) characters in the output format.
Standalone: String temp="Buôn Ma Thuột";
System.out.println("Test Buôn Ma Thuột:"+temp);

out put:Test Buôn Ma Thuột:Buôn Ma Thuột
Web application:
protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
response.setContentType("text/html;charset=ISO-8859-1");
//response.setContentType("text/html;charset=UTF-8");
String temp="Buôn Ma Thuột";
System.out.println("Test Buôn Ma Thu?t:"+temp);
}
out put:
Test Buôn Ma Thu?t:Buôn Ma Thu?t
i changed the content type in the two ways
a.UTF-8
b.ISO-8859-1
Even though i wont get required output.

Kindly help me.

[Vijay Tidake]

Hi,

While displaying the charters in other languages you have to first convert them in UTF-8 encoding

use native2ascii converter in <JAVA_HOME>/bin to do so.

Thanks

[rajkumar jayavel]

Thanks for your post.

Could you post the native2ascii with example?

[Vijay Tidake]

Hi,

consider you want to show the text "Name" in arabic.

First take two text files utf.txt and arabic.txt

convert all text in arabic.txt to utf-8 by using native2ascii convertor using command native2ascii -encoding UTF-8 arabic.txt utf.txt

Put the utf-8 text in properties file and shoe all your messages from properties file.

1.messages.properties
name=\u004 \u3423 \u3322 \u3422

Hope this helps you.

Thanks

[rajkumar jayavel]

Thanks vijay
My requirement is :

Fetch from data base as "Buôn Ma Thuột" while getting from the result set its showing as "Buôn Ma Thu?t" [Servlet call]
Finally i have to append the text as drop down values.

Now its showing as Junk(?) characters .

[Vijay Tidake]

Hi,

Try this one.Its showing the string as it is in servlet.

package demo;

import java.io.IOException;
import java.io.PrintWriter;
import java.io.UnsupportedEncodingException;
import java.util.LinkedList;
import java.util.List;
import java.util.Locale;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**
 * Servlet implementation class Demo
 */
public class Demo extends HttpServlet {
	private static final long serialVersionUID = 1L;

	/**
	 * @see HttpServlet#HttpServlet()
	 */
	List<String> list = new LinkedList<String>();

	public Demo() {
		super();
		String str = "Buôn Ma Thuột";
		byte[] utf8Bytes = null;
		try {
			utf8Bytes = str.getBytes("UTF-8");
			String roundTrip = null;
			roundTrip = new String(utf8Bytes, "UTF8");
			list.add(roundTrip);
		} catch (UnsupportedEncodingException e1) {
			e1.printStackTrace();
		}

	}

	/**
	 * @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse
	 *      response)
	 */
	@Override
	protected void doGet(HttpServletRequest request,
			HttpServletResponse response) throws ServletException, IOException {
		doPost(request, response);
	}

	/**
	 * @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse
	 *      response)
	 */
	@Override
	protected void doPost(HttpServletRequest request,
			HttpServletResponse response) throws ServletException, IOException {

		response.setCharacterEncoding("UTF-8");
		response.setLocale(new Locale("nl"));
		PrintWriter out = response.getWriter();
		out.write(list.toString());

	}
}

Hope this helps.

Thanks

[rajkumar jayavel]

Thanks Vijay

its working fine for me.Jai Ho!!!
Thank you very much.......!!!

[Paul Clapham]

Perhaps it works, but there's a lot of useless code. For example you could replace this:

    public Demo() {  
        super();  
        String str = "Buôn Ma Thuột";  
        byte[] utf8Bytes = null;  
        try {  
            utf8Bytes = str.getBytes("UTF-8");  
            String roundTrip = null;  
            roundTrip = new String(utf8Bytes, "UTF8");  
            list.add(roundTrip);  
        } catch (UnsupportedEncodingException e1) {  
            e1.printStackTrace();  
        }  
  
    }  

by this:'

    public Demo() {  
        super();  
        String str = "Buôn Ma Thuột";  
        list.add(str);  
    }  

The important line of code is this one:

response.setCharacterEncoding("UTF-8");

Pretty much everything else in the posted code is irrelevant.

[rajkumar jayavel]

Yes Paul...
i have tried without ut8byte converion
You are absolutely correct.

[rajkumar jayavel]

I found the solution for my problem.While executing this vietnamese characters are recognized.

CREATE FUNCTION [dbo].[FN_CONVERT_UTF8_2_VNI](
@P_INPUT as nvarchar(1000) = null
) RETURNS nvarchar(1000)
AS
BEGIN
DECLARE @P_OUTPUT nvarchar(1000)
SET @P_OUTPUT = @P_INPUT

SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'Õ', 'OÕ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'EÕ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'EÏ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'EÅ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'EÄ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'Ù', 'UØ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'É', 'EÙ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'Æ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'Ó', 'OÙ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'I', 'Ó' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'Ò', 'OØ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'Ò' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'OÛ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'Ã', 'AÕ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'EÃ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'AË' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'Á', 'AÙ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'EÁ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'À', 'AØ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'EÀ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'AÛ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'AÏ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'AÉ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'AË' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'EÛ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'AÜ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'È', 'EØ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'AÁ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'AÀ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'AÅ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'AÃ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'AÄ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'AÈ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'Ú', 'UÙ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'AÚ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'Ð', 'Ñ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'UÏ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ÖÙ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ÖØ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ÖÛ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'U', 'UÕ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ÖÏ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'Ý', 'YÙ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'YØ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'YÛ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'YÕ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'Î' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'OÏ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ÖÕ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'OÁ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'OÃ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'OÀ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'UÛ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'OÅ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'OÄ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'Â', 'AÂ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'Ô', 'OÂ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ÔÙ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ÔØ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ÔÛ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ÔÕ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ÔÏ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'Ê', 'EÂ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'U', 'Ö' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'A', 'AÊ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'O', 'Ô' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'õ', 'oõ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'eõ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'eï' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'à', 'aø' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'eà' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'eå' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'eä' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'ù', 'uø' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'é', 'eù' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'æ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'ó', 'où' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'i', 'ó' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'ò', 'oø' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ò' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'oû' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'ã', 'aõ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'eã' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'aë' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'á', 'aù' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'eá' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'oà' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'aû' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'aï' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'aé' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'aë' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'eû' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'aü' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'è', 'eø' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'aá' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'aà' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'aå' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'aã' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'aä' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'aè' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'ú', 'uù' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'aú' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'd', 'ñ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'uï' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'öù' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'öø' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'öû' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'u', 'uõ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'öï' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'ý', 'yù' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'yø' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'yû' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'yõ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'î' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'oï' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'öõ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'oá' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'oã' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'uû' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'oå' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'oä' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'â', 'aâ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'ô', 'oâ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ôù' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ôø' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ôû' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ôõ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'?', 'ôï' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'ê', 'eâ' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'u', 'ö' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'a', 'aê' )
SET @P_OUTPUT = REPLACE( @P_OUTPUT COLLATE Latin1_General_BIN, N'o', 'ô' )

RETURN @P_OUTPUT
END

GO


update TB_PPBranchOffice set BranchOffName=dbo.FN_CONVERT_UTF8_2_VNI(BranchOffName)