Sorry to bother you with this topic again, but I can't find information I need.
So, thanks your explanations in other topics, I was able to index forum without problems. However, I want to tweak it a bit - basically all I want to index from forum is content pages - that is viewtopic.php?p=. I can skip all other pages without problems, but to index all topics, I have to scan viewforum.php?f= and others. So probably I need another entity here - an ability to specify pages, which will be scanned for links, but not included in index. Hopefully you can understand my explanation
The reason for this approach is that only topic pages have the information user wants - if (s)he searches for something found in forum topic, there is no reason to display forum section - I want user to see topic directly.
Alternative approach can be including all pages to index, but an ability to skip some pages from search results - this would also help.
So, is there something that could help me?
Thank you in advance,
[For other readers of this post]. For a summary of the general issues and solutions to index PHPBB forums see this FAQ question.
The following might be a partial solution to your tweaking issue.
When in Spider mode:
Clicking on the “More >>” button will bring up a Advanced spider URL options window which allows you to add more spider URLs or specify advanced spider crawling options. This is particularly helpful when indexing across multiple websites or domains. With each spider URL you can specify one of the following options:
1) Index page and follow internal links: index the content of the specified page and follow any internal links (links to pages beginning with the base URL) found.
2) Index page and follow internal and external links: index the content of the specified page and follow any internal and external links. However, external links are only followed up to one level. For example, an external page linked from an internal page is scanned, but an external page linked from an external page is not.
3) Index single page only: index the content of the specified page and not follow any of the links found on it.
4) Follow links only: only follow the links found on this page but will not index any of the page content.
So if you know the URLs to all the forum sections in advance you could enter them into the URL list with option 4).
Another solution would be to modify the PHPBB code to insert and tags around the text you don't want indexed. This might be better solution if you have some small amount of PHP coding experience.
Thank you for replay, David!
Is follow links only mode will be able to index any found pages? If not, what is the reason for this mode?
Aslo, do I understand right, that ZOOMSTOP option will stop indexing, but will continue finding links, which only additional ZOOMSTOPFOLLOW can stop? If so, this is what I need - just confirm this fact, please.
Yes follow links only will not index the start page but will index pages that are linked to the start page.
Yes, this is the way ZOOMSTOP works. More details are at the link above and in the users guide.