• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • paul wheaton
  • Jeanne Boyarsky
  • Ron McLeod
Sheriffs:
  • Paul Clapham
  • Liutauras Vilda
  • Devaka Cooray
Saloon Keepers:
  • Tim Holloway
  • Roland Mueller
Bartenders:

Culprit for non-ascii (chinese, japanese) search problem

 
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I finally identified the chinese search problem.

My environment first:
Tomcat 4.1
Jdk 1.5
Debian sarge (LANG=en.US)
JForum 2.1.6

Sympton, searching chinese not returning desired result.

The principle cause of that is the submision method for search. JForum uses "GET" for search which naturally is just for ascii data. Unless backend explicitly do the convertion, GET would only work for ascii char set.

The solution is to use POST instead of GET for search. My test result shows it works. You only need to change search.htm, just replace GET with SET, no need to recompile.

The alternative is to do conversion in the backend but that involves code change and more risky - both from my experience and others' suggestions. There's much debate on this issue, I don't want to get into details here.

Another observation rather than problem is, JForum has a minimum length for a word to be indexed, which means it becomes searchable. So if you have a short word, it might not be searchable - that's a design rather than a bug. The current setting is

search.min.word.size = 3

So if you have a short message like ��܏�, it won't show up in the search result.

[originally posted on jforum.net by luorihui]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
By the way, I tested on both mysql and postgres, they worked fine.

Versions:
Postgres 8.1
Mysql 4.1
[originally posted on jforum.net by luorihui]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thank you very much for your repport.

In fact, POST was the initial setup for the search form, but we moved to GET to allow users to copy the url and paste in somewhere else.

Rafael
[originally posted on jforum.net by Rafael Steil]
 
Power corrupts. Absolute power xxxxxxxxxxxxxxxx is kinda neat.
Smokeless wood heat with a rocket mass heater
https://woodheat.net
reply
    Bookmark Topic Watch Topic
  • New Topic