How to read Franch from ResourceBundle

[Bruce Jin]

I am working on a English language PC. I have a message_fr_FR.properties file and some franch entries:

user-is-disabled=utilisateur est désactivé

When I read it from Java code I get “utilisateur est désactivé”

The code looks like this:

ResourceBundle messages = ResourceBundle.getBundle(“message”, aLocale);

String message =  messages.getString(“user-is-disabled”) ;

Why?
Thanks

[Bear Bibeault]

Looks like a character set issue. Make sure everyone is talking UTF-8.

[Bruce Jin]

Thanks Bear.

How to Make sure everyone is talking UTF-8?

I know the property file is encoded UTF-8.

Thanks.

[Bruce Jin]

How to Make sure everyone is talking UTF-8?
Thanks!

[Paul Clapham]

Looks to me like something is encoded in UTF-8 but actually read in some other encoding.

You said your properties file is encoded in UTF-8. However properties files are assumed to be encoded in ISO-8859-1 (as the API documentation states).

Which confirms my impression.

[Bruce Jin]

Paul Clapham wrote:Looks to me like something is encoded in UTF-8 but actually read in some other encoding.

Yes. The question is how to actually read in UTF-8?

Thanks!

[Paul Clapham]

No, that isn't the question. Or that's only one of the possible questions.

The answer to one possible question is to follow the rules and use ISO-8859-1 as the encoding for your properties file. Note that it's possible to do that for French text without having to mess about with Unicode escapes.

The answer to that question might be to not use ResourceBundle, which I expect follows the existing rules for properties files, and to use various Java 6 features which allow you to specify the charset while reading a properties file in your own substitute for ResourceBundle. Or it might be to use an XML-formatted properties file, for which the charset does default to UTF-8, if a ResourceBundle could use that.

[Bruce Jin]

For now I use a ResourceBundle.Control to solve to problem.
Thanks.

[Paul Clapham]

Aha! A real solution! Would you care to post some details about it? I'm sure you aren't the only one burned by Sun's short-sighted decision to force the use of ISO-8859-1.

[Bruce Jin]

This is the Control class:
Thanks.

public class MyControl extends ResourceBundle.Control {

	public List<String> getFormats(String basename) {

		if (basename == null) {
			throw new NullPointerException();
		}
		return Arrays.asList("properties");

	}

	@Override
	public ResourceBundle newBundle(String baseName, Locale locale, String format,
			ClassLoader loader, boolean reload) {

		if (baseName == null || locale == null || format == null || loader == null) {
			throw new NullPointerException();
		}
		
		ResourceBundle bundle = null;

		if (format.equals("properties")) {

			String bundleName = toBundleName(baseName, locale);
			String resourceName = toResourceName(bundleName, format);
			InputStream stream = null;

			try {
				if (reload) {
					URL url = loader.getResource(resourceName);
					if (url != null) {
						URLConnection connection;
						connection = url.openConnection();

						if (connection != null) {
							connection.setUseCaches(false);
							stream = connection.getInputStream();
						}
					}
				} else {
					stream = loader.getResourceAsStream(resourceName);
				}

				if (stream != null) {
					InputStreamReader is = new InputStreamReader(stream, "UTF-8");
					bundle = new PropertyResourceBundle(is);
					is.close();
				}
			} catch (IOException e) {				
				e.printStackTrace();
			}
		}

		return bundle;
	}

[Rob Spoor]

It's definitely an encoding issue. I just wrote a little test program; if the properties file is encoded using ANSI I get the expected output, but if I save it using UTF-8 I get the mess you got. I also get the same results if I use java.util.Properties with a FileReader with the system encoding. Only if I switch that do I get the actual content.

So that's what we need to do for ResourceBundle as well. ResourceBundle itself doesn't have any methods / constructors to help us with, so we have to go to ResourceBundle.Control. I've checked the source (available in src.zip inside the JDK folder), and it uses a PropertyResourceBundle combined with an InputStream. And guess what - this uses Properties.load with an InputStream, thereby getting the same problem as I got with a direct Properties object.

The solution is obvious - create a new ResourceBundle.Control subclass, and override newBundle to use a Reader instead of an InputStream. The following is my quick (but tested and working) attempt:

public class EncodedControl extends ResourceBundle.Control
{
    private String encoding;

    public EncodedControl(String encoding)
    {
        this.encoding = encoding;
    }

    @Override
    public ResourceBundle newBundle(String baseName, Locale locale, String format, ClassLoader loader, boolean reload)
           throws IllegalAccessException, InstantiationException, IOException
    {
        if (!format.equals("java.properties"))
        {
            return super.newBundle(baseName, locale, format, loader, reload);
        }
        String bundleName = toBundleName(baseName, locale);
        ResourceBundle bundle = null;
        // code below copied from Sun's/Oracle's code; that's their indentation, not mine ;)
		final String resourceName = toResourceName(bundleName, "properties");
		final ClassLoader classLoader = loader;
		final boolean reloadFlag = reload;
		InputStream stream = null;
		try {
		    stream = AccessController.doPrivileged(
			new PrivilegedExceptionAction<InputStream>() {
			    public InputStream run() throws IOException {
				InputStream is = null;
				if (reloadFlag) {
				    URL url = classLoader.getResource(resourceName);
				    if (url != null) {
					URLConnection connection = url.openConnection();
					if (connection != null) {
					    // Disable caches to get fresh data for
					    // reloading.
					    connection.setUseCaches(false);
					    is = connection.getInputStream();
					}
				    }
				} else {
				    is = classLoader.getResourceAsStream(resourceName);
				}
				return is;
			    }
			});
		} catch (PrivilegedActionException e) {
		    throw (IOException) e.getException();
		}
		if (stream != null) {
		    try {
// CHANGE HERE
//			bundle = new PropertyResourceBundle(stream);
			Reader reader = new InputStreamReader(stream, encoding);
			bundle = new PropertyResourceBundle(reader);
// END CHANGE
		    } finally {
			stream.close();
		    }
		}
        // and to finish it off
        return bundle;
    }
}

//ResourceBundle messages = ResourceBundle.getBundle(“message”, aLocale);
ResourceBundle messages = ResourceBundle.getBundle(“message”, aLocale, new EncodedControl("UTF8"));

I'm actually disappointed that Sun/Oracle haven't provided such a class themselves.

[Rob Spoor]

Wow, I've spent half an hour testing with the default control, getting to the cause, writing the class, testing it and posting it...