PHP / CGI script returns no results on Apache RLimitMEM & RLimitCPU

[wrensoft]

A note to Zoom users that use the PHP or CGI version of Zoom on servers that run Apache.

Several users have reported a problem where they do a search and get no results. In some cases the search form is still displayed at the top of the page, in other cases you get a completely blank page.

After investigation it seems that some shared hosting companies are severly limiting the amount of RAM and CPU available to PHP and CGI scripts.

They can do this by setting the "RLimitMEM" & "RLimitCPU" directives in the Apache configuration for the server. Setting these values will cause Apache to kill any script after it uses more than a certain amount of RAM or CPU time.

These values are normally set in the Apache httpd.conf file. (but can also appear in the srm.conf, access.conf and virtual host files)

See,
http://www.apacheref.com/ref/http_core/RLimitMEM.html
http://www.apacheref.com/ref/http_core/RLimitCPU.html

The timeout setting is also worth looking at.
http://www.apacheref.com/ref/http_core/Timeout.html

The solution is to get you hosting company to set reasonable limits for these values. Of course they might not like this becuase it means you are going to be using more of their CPU time and RAM.

The larger your index files in Zoom, the higher these limits will need to be. Example limits for a medium sized site might be,
RLimitMEM 32097152 32097152 #32 Meg of RAM
RLimitCPU 20 60 #60 seconds of CPU time

But before you start sending your hosting company nasty E-Mails check everthing else is correct.
- Check the the indexing process was successful
- Check that all files are on the server
- Check that you can run other small PHP scripts.

Note that if you have this problem, then there will be no error message of any sort, but you will see the search form. This is becuase the script was suddenly terminated. If you see an error message then these settings are probably not the cause of your problem.

--------
David
Wrensoft

Update: Oct 2005: If you are able to confirm with your hosting company that CPU run time is the problem and you are using PHP, then consider switching to the CGI option which will use less CPU time and give faster searching.

[Joe Everett]

Joe Everett
tbaptist.com

I have this very problem where there are no search results posted on the search form.

The search engine is working fine one day, and then a few days later it isn’t.

There is no apparent problem with the search files. They have been uploaded more than once, but no change. The search engine (with these search files) works fine on my home computer using a server application.

I showed the RLimitMEM and RLimitCPU posting to my server webmaster, but he says that cannot be the problem, that the server values are much more than enough.

He says they are using this configuration (which, admittedly, means little to me):

Dul 2.8 w/4 GB ram

Fedora Core 2

php 4.3.8

I am using Zoom Professional 3.1, I think (I don’t have it here for checking); I bought it last May.

The server webmaster suggested that I enable error-logging --- how do I do this, and will it help since there are no errors reported?

He also suggested that the search-engine may not work with php 4.3.8.

Any suggestions?

Regards, Joe Everett

[wrensoft]

I just went to your web site and did a search,

http://www.tbaptist.com/zoom/search.php?zoom_query=bible

387 results were found and displayed correctly.

So I assume that you have now fixed your problem becuase it seems to be working fine?

--------
David

[Anonymous]

"So I assume that you have now fixed your problem becuase it seems to be working fine?"

Well, so it is!

It wasn't working this morning, but I guess my server company figured out the problem.

Thanks for checking it out anyway; greatly appreciated.

Regards, Joe Everett

[wrensoft]

There are still a few people seeing this issue. So I wanted to post some additional details here.

There are a number of cheap hosting companies that paid peanuts & employ monkeys to do support. These companies often deny there are any limits on their servers.

This is a very simple script which should reach a count of 9,000,000. If it stops before this then the script's execution must have been terminated by something on the server. This would indicate that it is a problem independent of the Zoom search script.

<?php
print("Begin test loop to allocate memory for 9,000,000 integers\n");
for ($i = 0; $i < 9000000; $i++)
{
$dict[$i] = $i;
if ($i % 5000 == 0)
print ("allocated memory for " . $i . " integers. ");
}
print("End test loop\n");
?>This can help you prove to the monkey that there is a limit on their servers.

EDIT: Previously this script counted to 2,000,000. But we edited this post a couple of years after it was written, and increased it to 9,000,000 to simulate heavier RAM and CPU usage.

----
David

[hoosierwebdesign]

Thanks so much for the software (at a great price)
After running the script and only getting allocated memory for 130000 integers, (On our dedicated server) I was able to look at the limites tab and noticed it had changed from the default of 1024kb to 10240kb.. That did not work :(

So as we have the search running on 2 different parts of our site, I loaded the settings fromt he working search, and just changed the web folder.. reran the index and now I do get how many results it found but nothing listed under that.

I now need to look into changing the limits but have no clue how to access the httpd.conf file.

I am just not sure why it doesn't work on one part of the site but it does on the other... (php shopping cart side works, html old site does not(but did)



Thanks again for the professional and quick reply to my email,
Jason Miller

[hoosierwebdesign]

uninstalled/reinstalled and now it works again..

Strange!

[wrensoft]

I glad it is working again, becuase to be honest, I was a bit confused by the description of the problem.

------
David

[wise_mike]

Hi there

Where do I put that code mentioned above? at which file? or as a new file or what?

Thanks in advance

[wrensoft]

Yes. You make a file in Notepad called something like test.php, paste in the simple test script above, save it. Upload to your server and run the script.

It should print a bunch of lines like,

allocated memory for 5000 integers
...
allocated memory for 2000000 integers
End test loop

If the last line, "End test loop", never prints out you have hit a server limit.

----
David

[wise_mike]

Is there a way that I could host the search fiiles on another server that direct to my current server?

Thanks

[wrensoft]

Yes, you can upload the search files to any server that meets the requirements. The search results will still point back to the original documents on the original server.

-----
David

[glenna1984]

I ran the php test file and it stopped at 230000.

My hosting company is 1and1. They will not tell me what the RLimitMem and RLimitCPU is -- so far.

However, they maintain I can control this with .htacces.

Do you agree, and if so, what are the commands?

[Ray]

I believe it is possible to modify RLimitMEM and RLimitCPU via .htaccess if the Apache server is configured to allow this. However, this is a highly unlikely setting unless you are on a dedicated server. This would otherwise mean that any account on your shared server may be requesting to use 100% CPU or all available RAM, and the server is unlikely to handle this. If they maintain that you can control this, they should be able to tell you how to do so. The syntax should otherwise be the same as the setting in the php configuration file.

Alternatively, they might be thinking of the memory limitation for PHP which is more commonly available for user control in .htaccess as described in this FAQ (http://www.wrensoft.com/zoom/support/faq_php.html#php_outofmemory). This limit is applied prior to the Apache limit, which means that even if you have this limit allowing for 32MB of memory, if there is an Apache RLimitMEM set to 4MB, you will still be restricted to 4MB. It would be worth checking however, if it is the PHP limit that you are running into or the Apache one. The PHP limit should normally report an error message (eg. ""Fatal error: Allowed memory size of 8388608 bytes exhausted..." or similar) either on the page itself or in the server logs.

[carloant1953]

I have tested script test.php , but obtain this error message:

Warning: Unexpected character in input: '\' (ASCII=92) state=1 in /var/www/web236/html/test.php on line 7

Parse error: parse error, unexpected T_PRINT in /var/www/web236/html/test.php on line 8

thank you

[Ray]

The code got line wrapped by the forum software, so some of the code appeared on a second line when it should have been on the same line. This may have been the cause of the error you are seeing.

I've just edited the post and fixed up the wrapping. Copy and paste it again, exactly as it is. It should work now.

[isd]

hello,
i have one question regarding the results, please. so if my last line ends like:
"integersallocated memory for 1995000 integersEnd test loop"
it means on this server configuration my zoom_wordmap.zdat which weights 5839292 is limited to nearly 30% ?

thank you very much for help in advance
ewa

[wrensoft]

If you see the "End test loop" message, Then no RAM or execution time limit was hit. Meaning that the server allowed the test script enough RAM to hold 2 million integers (which is about 8MB of RAM).

You could try changing the test script to try a larger value. e.g.

for ($i = 0; $i < 10000000; $i++)
This corresponds to about a >40MB RAM requirement.

You can't make any direct comparision to the size of the zoom_wordmap.zdat file as not all of the zoom_wordmap.zdat is held in RAM at any one time.

[isd]

ok, thank you very much David, that helps to understand much better, btw. in my case (http://www.wrensoft.com/forum/showthread.php?t=2404) the memory limit can be excluded as reason of those occurrences?

thank you very much for help in advance
ewa

[Ray]

The PHP test can be used to show that the problem is evident, but it is not conclusive to prove that there is no limit put in place.

The reason is that it depends on how PHP is configured on the server. If the PHP scripting engine is setup as a CGI application which is forked by Apache, then the RLimit* operatives will apply and the PHP script can prove this. If they run within httpd however, the limits would not apply to the PHP engine.

[austenozzy]

Just a quick note to say thanks for this thread, it solved the problem for me!

I called rackspace (we're lucky enough to have a dedicated server) and they altered the memory available to PHP from 8MB to 32B and it solved it straight away.

Cheers,

Austen.

[wrensoft]

This thread was started 4 years ago. Even 32MB isn't much by today's standards (in 2009 with increasing volumes of data).

PHP as a language isn't very efficient in using RAM, and so 64MB or 128MB is more reasonable for PHP when typical servers have 2GB or 4GB available.

[wrensoft]

I am going to start making a list of known bad hosting companies. These are the ones who in my opinion that so severely restrict your usage of PHP and CGI that it makes their service close to useless. (You could even say fraudulent, as they don't announce all these limits in advance of taking your money).

Here is the list of hosts that are known to have this problem (they should be avoided in my opinion).

Yahoo hosting
Internet America (If you ask nicely to the right person, they may fix it for your server)
EarthLink (at mid 2009, they also use very slow hardware, so really avoid these guys)
GoDaddy
1&1 hosting
Infinology