PDA

View Full Version : noindex and no follow

JCF1976
12-13-2006, 06:51 PM
Is there a way to get Zoom to obey:

<meta name="robots" content="noindex">and this one too:

<meta name="robots" content="no follow">...I understand that one may put Zoom tags on pages, but if I am crawling sites that are not my own, then I have no control over inserting Zoom tags. Furthermore, if Zoom does not obey noindex and no follow, when crawling other people's sites, then my IP could be banned from crawling their sites. That's not a good thing.

If there is not presently (in V5) an option to have Zoom obey noindex and nofollow, will this feature be added in V5.1 or V5.2?
Ray
12-13-2006, 11:18 PM
It is on our list of things to do. V5.1 is a likely candidate for this feature.


UPDATE
This feature was added in V5.1 of Zoom, 15/June/2007.
Zoom will obey meta robots tags for "noindex" and "nofollow", e.g.
<meta name="robots" content="noindex">
JCF1976
12-15-2006, 04:44 PM
It is on our list of things to do. V5.1 is a likely candidate for this feature.

Ray, sounds good! Also, I am wondering how to get the search engine to perform searches like:

site:mywebsite.com

on a search engine that has several different web-sites crawled an indexed together. Is this possible? Is this something that will come out in V5.1?
wrensoft
12-15-2006, 07:03 PM
At the moment it is not possible to do this. At least not without doing some modifications on the scripts. If there is enough demand we would implement it in a future release. But there hasn't been to date.
JCF1976
12-15-2006, 07:46 PM
At the moment it is not possible to do this. At least not without doing some modifications on the scripts. If there is enough demand we would implement it in a future release. But there hasn't been to date.

Well, add me to the list of those demanding it! :-) Kidding. Seriously, I think it should be a standard feature, especially if someone is crawling multiple sites.