PDA

View Full Version : Advanced Spider Options



InteriorHobbies
06-11-2009, 04:10 PM
Hello I am trying to import a number of start points and would like to set them to "Follow all links on this page only" I can not find the command for that option. I even set up one start point set it to "Follow all links on this page only" exported it and here is the results

http://www.interiorhobbies.com/items.asp?Cc=ADAPTOR&iTpStatus=1&Tp=&Bc=, INDEX_AND_FOLLOW, http://www.interiorhobbies.com/

Thank you

Mike Eaket

Ray
06-12-2009, 01:05 AM
Seems like there's two problems here that you found.

First, the Users Guide is missing the documentation for this feature. We'll have to update it to include it.

The "Follow all links on this page only" option is specified as "FOLLOW_ALL" in the import/export file.

Second problem is the bug that the "Export" feature is not exporting the "follow all links..." option properly.

The good news is that it does correctly import the option "FOLLOW_ALL".

We'll fix the documentation and bug in the next release (V6.0 build 1016). Thank you for bringing it to our attention.

InteriorHobbies
06-12-2009, 04:10 AM
I found this in the Sticky : New features in V6

New spider option: URLTYPE_FOLLOW_ALL (follow all pages to one level for a start point without indexing the start page).

I belive this is the command for "Follow all links on this page only"

Mike

I just want to add this is a great product , it is really going to improve my sites search.

Ray
06-12-2009, 07:12 AM
It should be "FOLLOW_ALL" as mentioned above. We'll fix the post too.