Incorrect display of UTF-8 characters (traditional Chinese)

[John Tsang]

I set language encoding in Zoom Indexer configuration to utf-8. All multi-byte characters are displayed incorrectly in search results. Adding a preprocessing directive <%@ CODEPAGE=65001%> or <%@ CODEPAGE=1252%> to the top of my ASP page does no search results come up. But I found that once I change the server regional setting from traditional chinese to US English, everything ok. However change the production server regional setting may affect other issues. I think there is a work around solution?

The case just like http://www.wrensoft.com/forum/showthread.php?t=2284

Setting:
My intranet website using IIS 5.0 with dotnet 1.1
Server regional settings is Traditional Chinese
Zoom Search Engine 6.0 Enterprise Edition (6.0.101

John

[Ray]

It's surprising you're seeing this problem as I believe we've done everything we can to avoid it. However I did notice you're using IIS 5.0 which is rather old and, I believe, no longer supported by Microsoft. We haven't tested on an IIS 5.0 server for a while now (it has been well and truly superseded -- the latest version is IIS 7.0 and the oldest version we've seen for some time now is IIS 5.1) so the behaviour might be related to a quirk of IIS 5.0 specifically.

Do you have another server (test/development box or otherwise) running a newer OS/IIS that you have access to? See if the same problem occurs there. It would be difficult for us to try and replicate your server environment.

As a last resort, there is always the option to switch to the CGI platform (provided you have adequate server permissions to enable this) which would avoid all these ASP scripting environment issues.

[John Tsang]

Also I tested in my PC (windowXP IIS 5.1) have same problem too. The UTF-8 characters (traditional Chinese) is correctly display but no result come up after add the <%@ CODEPAGE=1252%>. If I don't add the <%@ CODEPAGE=1252%>, it have search results. Is it not support IIS 5.1 too ?

John

[Ray]

Are you saying that if you don't add the CODEPAGE directive, the chinese characters are displayed and searchable in IIS 5.1? Then there is no problem?

You don't need to add the CODEPAGE directive if there's no problem.

Or maybe I'm misunderstanding you.

[John Tsang]

I means

If don't add the CODEPAGE directive, it can searchable but all chinese characters display incorrectly.

If add the CODEPAGE directive, it cannot searchable (no results) but all chinese characters display correctly.

I tested both IIS 5.0 (Window2K) and IIS 5.1 (WindowXP) have same outcome.

[Ray]

Is the server available online? Can you give us a URL to the search page? The searching problem might be due to something else (e.g. if you have not enabled substring matching).

We're no longer running IIS 5.x so we can't easily reproduce the problem here. If you can replicate this problem in IIS 6, we can take a look (email us your index files and your ZCFG configuration file).

[John Tsang]

Is the server available online? No.

I just tested IIS 6.0 (Window2003) have same outcome. I already email to you with the index and configuration files.

John