Javascript feature suggestion

[iantresman]

I'm guessing that the Javascript interface has a unique word limit because the resulting zoom_index.js file gets very large.

Could the file zoom_index.js be subdivided into 26, for each letter of the alphabet, or for faster use, into more files, (eg. Ba.. Be.. Bi.. etc).

Regards,
Ian Tresman

[David]

Yes, there is a limit with Javascript that is not present with ASP, PHP & CGI.

The Javascript limit is not set by Zoom, but by how much RAM the browser you are using allows Javascript to allocate. IE6 seems to allow about 10MB of RAM to be used. Irrelevant of how much RAM you have available.

Sub-dividing the dictionary file (which is only part of index data) doesn't solve the problem. Some searches like a wild card search require the entire dictionary to be available.

[iantresman]

1. It's not a simple matter of wildcard searches of the form A* would search the "A file", and of the form "*ing" would search each letter file one after another?

2. Could you not tokenize the zoom_index.js file, for example, storing the indexes in hexadecimal, or 2-byte or 3-byte codes?

Regards,
Ian Tresman

[David]

Searching each letter file one at a time would work in most other programming languages. But not in Javascript. You can only include files, there is no un-include function in the language to remove them from memory. In fact no memory deallocation function at all. You also can't just read the files from disk. Any attempt to directly access the hard disk from Javascript results in IE popping up security warnings.

You also can't deal with binary coded files in Javascript. Mainly because you can't read them from the hard disk (without getting security warnings). But also for other reasons like Javascript not coping with 3 byte integers very well.

If you can prove us wrong by posting some code, that would be great.

We have written the same search script in Javascript, ASP, PHP and C++. Javascript is by far the worse language & our believe is that Javascript is really a ugly mess and certainly not designed for the serious data processing we are trying to do (but we do what we can within the limits of the language).

[iantresman]

If you can prove us wrong by posting some code, that would be great.

http://www.unitedscripters.com/index...pts/math4.html

This script is a number base converted.
For example, to store 5-digit base-10 numbers (plus single spacer) to 3-digit base 64, you'd use:

<font size=+1 face="courier"><b>
<script language="JavaScript">
a=baseConverter(10, "1234567890abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLM NOPQRSTUVWXYZ$", '99999', '', 1, 1)
document.write (a)
</script>

This would let you reduce your index file by about 40%.
The largest base is base 94, 128 - 34 (control characters) which is all the printable characters ascii 33 to ascii 126.

Regards,
Ian Tresman

[David]

We were refering to JS code that would read a file from disk (without security warnings)

But to address your suggestion. Your maths is wrong on several counts.

1) Moving from 6 to 4 characters (including the space) is only a 33% saving

2) You assume all our index files are all 5 digit numbers but in fact there is a lot of ASCII text. Vastly reducing the benefit.

3) A lot of the numbers used are shorter than 5 digits. For example, the benefit of converting a 2 digit base 10 number to base 64 is 0%. Further diluting any possible benefit.

But doing a base conversion on the numbers would also slow down the already slow execution of Javascript.

Plus it is likely that in some cases extra RAM would be required to hold the numbers in two forms, base 64 and post conversion base 10. Which is exactly what we are trying to avoid.

I think the net benefit would be a gain of a couple of percentage points in capacity in exchange for slightly slower execution speed.