JSF and Character Sets (UTF-8 mainly)

[Jarno Walgemoed]

Hi all,

This question might have been asked before, but I'm going to ask it anyway because I'm completely puzzled by how this works in JSF.

Let's begin with the basics, I have an application running on an OC4J servlet container, and am using JSF 1.1 (MyFaces). The problems I am having with this setup, is that it seems that the character encodings I want the server/client to use are not coming across correctly. I'm trying to enforce the application to be UTF-8, but after the response is rendered to my client, I've magically been reverted to ISO-8859-1, which is the main character set for the netherlands. However, I'm building the application to support proper internationalization; which means I NEED to use UTF-8.

I've executed the following steps to reach this goal:

- All JSP files contain page directives, noting the character set:

<%@ page pageEncoding="UTF-8" contentType="text/html; charset=UTF-8" %>

I've checked the generated source that comes from the JSP's, it looks as expected.

- I've created a servlet filter to set the character set directly on the request and response objects:

    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) throws IOException, ServletException {
        // Set the characterencoding for the request and response streams.
        req.setCharacterEncoding("UTF-8");
        res.setContentType("text/html; charset=UTF-8");        
        
        // Complete (continue) the processing chain.
        chain.doFilter(req, res);  
    }

I've debugged the code, and this works fine, except for where JSF comes in. If I use the above situation, without going through JSF, my pages come back UTF-8. When I go through JSF, my pages come back as ISO-8859-1. I'm baffled as to what is causing this. On several forums, writing a filter was proposed as the solution, however this doesn't do it for me.

It looks like somewhere internally in JSF the character set is changed to ISO. I've been through the sources, and I've found several pieces of code that support that theory. I've seen portions of code where the character set for the response is set to that of the request. Which in my case coming from a dutch system, will be ISO.

How can this be prevented? Can anyone give some good insight on the inner workings of JSF with regards to character sets in specific? Could this be a servlet container problem?

Many thanks in advance for your assistance,

Jarno

[Peter Goldstein]

Jarno,

I see a similar behavior and I'm not sure of the underlying cause. I can see some suspicious lines in the FaceletsViewHandler (I'm using Facelets) that lead me to suspect that the request character encoding is simply copied to the response writer. This is in JSF 1.1.

However if you're able to upgrade to JSF 1.2 there may be a solution. In 1.2 the ViewHandler interface includes a calculateCharacterEncoding(FacesContext) method. I'm assuming that this works similarly to the calculateLocale(FacesContext) in JSF 1.1. In which case you should be able to force the character encoding to UTF-8.

Not sure if that helps, but it seems interesting.

Regards,

Peter

[Jarno Walgemoed]

Hi,

Thanks for the input, really appreciate it. I however am not in a situation where I can switch to the next faces release. This is a bit of a pain, because the code you are describing surfaces in the ViewHandler code as well. It seems that the request is leading in determining the character encoding for JSF responses.

That's not just it though. I've not pinpointed where it happens yet, but somewhere inside the code the request's characterset gets altered. If I write a filter to set the request encoding to UTF-8 before any other application code is executed on the request, the request's character encoding still gets set to ISO-8859-1 somewhere down the line. So it looks like JSF is first setting the request encoding to ISO-8859-1, and then injects that encoding into the response. Odd behaviour, as far as I am concerned, and I truly hope it is configurable. However, whilst roaming the web, I haven't found any proof that it is indeed configurable, or even that the problem itself is real.

If anyone could give me a definate answer, I would truly appreciate this.

[R bhushanam]

Originally posted by Jarno Walgemoed:
Hi all,

This question might have been asked before, but I'm going to ask it anyway because I'm completely puzzled by how this works in JSF.

Let's begin with the basics, I have an application running on an OC4J servlet container, and am using JSF 1.1 (MyFaces). The problems I am having with this setup, is that it seems that the character encodings I want the server/client to use are not coming across correctly. I'm trying to enforce the application to be UTF-8, but after the response is rendered to my client, I've magically been reverted to ISO-8859-1, which is the main character set for the netherlands. However, I'm building the application to support proper internationalization; which means I NEED to use UTF-8.

I've executed the following steps to reach this goal:

- All JSP files contain page directives, noting the character set:

<%@ page pageEncoding="UTF-8" contentType="text/html; charset=UTF-8" %>

I've checked the generated source that comes from the JSP's, it looks as expected.

- I've created a servlet filter to set the character set directly on the request and response objects:

    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) throws IOException, ServletException {
        // Set the characterencoding for the request and response streams.
        req.setCharacterEncoding("UTF-8");
        res.setContentType("text/html; charset=UTF-8");        
        
        // Complete (continue) the processing chain.
        chain.doFilter(req, res);  
    }

I've debugged the code, and this works fine, except for where JSF comes in. If I use the above situation, without going through JSF, my pages come back UTF-8. When I go through JSF, my pages come back as ISO-8859-1. I'm baffled as to what is causing this. On several forums, writing a filter was proposed as the solution, however this doesn't do it for me.

It looks like somewhere internally in JSF the character set is changed to ISO. I've been through the sources, and I've found several pieces of code that support that theory. I've seen portions of code where the character set for the response is set to that of the request. Which in my case coming from a dutch system, will be ISO.

How can this be prevented? Can anyone give some good insight on the inner workings of JSF with regards to character sets in specific? Could this be a servlet container problem?

Many thanks in advance for your assistance,

Jarno

hi Jarno ,

Can you please tell me the solution for the same issue. I am also getting the same problem. I have the environment JSF1.1, oc4j904 container, in filter servlet added the servletRequest.setCharacterEncoding("UTF-8");
First time it's coverting to UTF-8, But from second time encoding to ISO-8859-1.



Thanks & regards
bhushanam.

[Micharel Kr]

ANYBODY SOLVES IT ?

[yigit darcin]

this can be helpful

http://www.jroller.com/mert/entry/utf_8_encoding_with_jsf

[Gerard Visser]

I've encountered this ghostly problem also. It's un (unwanted) effect that your webpages are "given through as having the standard ISO-code" by the hosting server because of a configuration file. A second provider won't help me with it, but a year ago a first provider came with this solution: change a file called .htaccess (yes, nothing before the dot), in that little file should be a line that reads "AddDefaultCharset Off" to send your webpages unchanged. For me this line was reading "On", so I changed it to "Off". For my website "microcosmos.nl" this did the trick. On another website I want to do the same thing, but cannot find OR UPLOAD the ".htaccess" file, and the provider doesn't want to help