Cannot index oscommerce site

[izzyshams]

I have site running creloaded backend and would like to index it using the Zoom Search engine sotware, but for some reason I can't index it. The site is coded in php, so the index is index.php

The log file says the following (I have removed my site name to prevent google indexing):

04/11/07 20:02:57 - Start indexing (spider mode)
04/11/07 20:02:57 - Maximum number of words: 15000
04/11/07 20:02:57 - Maximum number of files: 50
04/11/07 20:02:57 - Will scan files with extensions
04/11/07 20:02:57 - .php
04/11/07 20:02:57 - .asp
04/11/07 20:02:57 - .cgi
04/11/07 20:02:57 - .aspx
04/11/07 20:02:57 - .pl
04/11/07 20:02:57 - .php3
04/11/07 20:02:57 - Spider from: http://www.*******.com/index.php
04/11/07 20:02:57 - Web site URL: http://www.*******.com/
04/11/07 20:02:57 - Estimated RAM required during index process: 4576 KB
04/11/07 20:02:57 - Initiating HTTP session (thread #1) ...
04/11/07 20:02:57 - DL Thread #1, got URL (http://www.*******.com/index.php) off queue
04/11/07 20:02:57 - Downloading file http://www.*******.com/index.php
04/11/07 20:02:59 - Could not download file: http://www.*******.com/index.php (Internal server error)
04/11/07 20:02:59 - Initiating HTTP session (thread #2) ...
04/11/07 20:03:00 - No files found to spider from http://www.*******.com/index.php
04/11/07 20:03:00 - Indexing failed
04/11/07 20:03:00 - Waiting for threads to finish ...
04/11/07 20:03:00 - Cleaning up memory used for index data... please wait.
04/11/07 20:03:00 - Deleting wordmap data...
04/11/07 20:03:00 - Deleting presaved index data...
04/11/07 20:03:00 - Deleting pageinfo data...
04/11/07 20:03:00 - Deleting miscellaneous buffers...
04/11/07 20:03:00 - Deleting URL history...
04/11/07 20:03:00 - Finished cleaning up memory.


I've used other website search programs and they spidered the site fine. I really like the features of Zoom Search though and hopefully this won't be a deal breaker for me. Any suggestions?

[wrensoft]

The key line in the log is this one,
Could not download file: http://www.*******.com/index.php (Internal server error)

This means that the server failed or refused to serve up the page.

If you look at the page using IE, you'll probalby see the same HTTP Error 500 Internal server error.

Just to double check, I went to the osCommerce web site and indexed a few of their showcased web sites, without any problem.

[izzyshams]

The site works fine in IE as well as other browsers. Although I figured out what the problem was. It was that I had gzip compression enabled for my website. Once I turned that off, I was able to successfully index my site.
Thanks Wren!