Search Benchmarks

PHP vs ASP vs ASP.NET vs JavaScript vs CGI

Zoom provides four platform options for searching: PHP, ASP, ASP.NET, JavaScript, and CGI. These searching front ends processes the user search queries, searches through the index data, and displays the results on your website.

Below are some benchmarks we have made comparing the 5 different versions of the Zoom search scripts provided, so that you can see how Zoom performs on sites of certain sizes with the available scripting platforms.

In the tests below, we benchmarked several different scenarios:

  • Searching a small/typical website
    with 400 pages, and 266,479 total words (21,271 unique words).
  • Searching a medium website
    with 60,000 pages, and 2,590,269 total words (153,986 unique words).
  • Searching an large website (CGI and ASP.NET only)
    with 125,000 pages, and 6,193,693 total words (597,642 unique words).

Note: The CGI (C++) version can search sites larger than 60,000 pages and 300,000 unique words, but neither the PHP or ASP version is capable of this, so the largest site compared here is under this 60,000/300,000 limit.

We measured the performance of each of these sets of data with a range of search query test cases.

Test server specifications:

  • Intel i7 2.66 GHz
  • Windows 7 Ultimate (64-bit)
  • Western Digital 500GB SATA2 GreenPower, 7200 RPM
  • 4 GB of RAM
  • PHP 5.2.9-1 on Apache 2.2.11
  • ASP and ASP.NET tested on IIS 7.5.7000.0
  • CGI (C++) application tested on Apache 2.2.11
  • JavaScript tested on Internet Explorer 8.0.7000.0
  • Zoom Search Engine V6 (build 1013), Enterprise Edition

Searching a small/typical site

400 pages and 266,479 total words (21,271 unique words)

Time is in seconds to generate and display results (the smaller the values the better)

Single word queries 2 words 3 words Wildcard Exact phrase
Platform Test 1 Test 2 Test 3 Test 4 Test 5 Test 6 Test 7
PHP 0.0830 0.0997 0.0877 0.0973 0.1080 0.1180 0.1500
ASP 0.2083 0.1980 0.2137 0.1880 0.2083 0.3020 0.4273
JavaScript* 0.2653 0.3153 0.2577 0.3517 0.4843 0.4427 N/A*
ASP.NET 0.0180 0.0180 0.0187 0.0157 0.0190 0.0130 0.0150
CGI (C++) 0.0120 0.0117 0.0167 0.0130 0.0120 0.0130 0.0150
*Each JavaScript time includes an assumed download duration, to represent the downloading of the index files by the browser when it is hosted on a remote web server. This assumes certain bandwidth conditions. Note that JavaScript would be much faster when run directly from a local CD/DVD or hard disk. PHP, ASP, ASP.NET and CGI do not need the index files to be downloaded.

Benchmarking PHP vs. ASP vs ASP.NET vs Javascript vs Binary CGI (C++) on a typical site search
Test 1 to 5 are random queries involving single to multiple search words.
Test 6 involves the use of wildcards, and Test 7 is an exact phrase query.
Note that wildcard and exact phrase searches have a significant impact on speed.
JavaScript does not support exact phrase searches so it does not have a result for Test 7.

 

Searching a medium site

60,000 pages and 2,590,269 total words (153,986 unique words).

Time is in seconds to generate and display results (the smaller the values the better)

Single word queries 2 word 3 words Wildcard Exact phrase
Platform Test 8 Test 9 Test 10 Test 11 Test 12 Test 13 Test 14
PHP 0.8750 0.9763 1.0730 0.9817 1.1677 1.2930 1.0097
ASP 4.9580 4.9113 4.9947 4.9530 5.1197 6.0780 5.0780
JavaScript* 2.8230 5.4123 5.8960 5.7500 8.9010 6.5363 N/A*
ASP.NET 0.0763 0.0787 0.0823 0.0870 0.0970 0.0933 0.0810
CGI (C++) 0.1053 0.1083 0.1287 0.1040 0.1190 0.1353 0.1097

Comparing performance of PHP, ASP, ASP.NET and CGI (C++) on a medium sized site
Test 8 to 12 are random queries involving single to multiple search words.
Test 13 involves the use of wildcards, and Test 14 is an exact phrase query.
Note that the CGI (C++) and ASP.NET versions scales significantly better with larger sites.

Searching a large site

125,000 pages, and 6,193,693 total words (597,642 unique words).

Time is in seconds to generate and display results (the smaller the values the better)

Single word queries 2 word 3 words Wildcard Exact phrase
Platform Test 15 Test 16 Test 17 Test 18 Test 19 Test 20 Test 21
ASP.NET 0.3433 0.3370 0.3700 0.3490 0.3627 0.3853 0.3623
CGI (C++) 0.2587 0.2597 0.2683 0.2603 0.2737 0.2787 0.2580

ASP.NET vs binary (C++ compiled) CGI search benchmarking on a large site with over 6.1 million words
Test 15 to 19 are random queries involving single to multiple search words.
Test 20 involves the use of wildcards, and Test 21 is an exact phrase query.
Note that this test was designed for the CGI(C++) and ASP.NET platforms only.
The PHP, ASP, and JS versions can not be used to search this amount of data.

Benchmark Notes

  • Search script settings: 10 results per page, "any" search word, context descriptions (except for JavaScript), exact phrase matching and stemming enabled (except for JavaScript), highlighting is enabled, logging is disabled, and timing is enabled.
  • Operations are performed and measured on a local computer, and assume no network latency, or shared CPU processing load.
  • Assumes no graphics, or extraneous layout in the template file, nor any extra header or footer scripting
  • All test results are a calculated average of 3 runs. The first test for each site is performed after a fresh reboot and thus do not have the files cached.

Benchmark Conclusions

  • On average, the PHP version is faster than the ASP version, while the CGI (C++) version is more than 10 times faster than both PHP and ASP.
  • The PHP version is only slightly faster than the ASP version for smaller sites, but as the size of a site grows, the difference increases. This indicates that the PHP version scales better than the ASP version.
  • The CGI (C++) version performs the best and scales significantly better than the other versions as the size of a site increases (up to 70 times faster in some cases). The native ASP.NET Server Control also performs well. They are the only platform capable of searching through an enterprise size site of over 65,000 pages (as demonstrated in the "large site" test).
  • The JavaScript version produces the slowest results. This emphasises the fact that it should only be used exclusively for offline distributions (in which case it performs well because there is no download time), or as a last resort for small online website. Due to the technical restrictions of the JavaScript platform, it also has the least capable support of features (it does not allow context descriptions, exact phrase matching nor stemming), and can not be used on larger sites due to memory restrictions enforced by browsers.
  • The use of wildcards result in slower searches.
  • The use of exact phrase searches can result in slow searches, depending on the commonality of the words in the phrase.
  • There is a slight increase in the search delay when the index files are not already loaded in the system's cache (as expected).

Return to the Zoom Search Engine product page to find out more.