Home » Forum
  • If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

Announcement

Collapse
No announcement yet.

Spidering sites with user name and passwords, authentication

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Spidering sites with user name and passwords, authentication

    All Zoom versions can access password protected sites, but Zoom only supports the standard HTTP based web server authentication. For this type authentication the browser pops up a Windows dialog, in a new window, prompting for a username and password.

    If you use any other method to get and verify user names, passwords or other details it probably won't work. Or only work some of the time if the right cookies happen to be present.

    There are some alternatives however. For example.

    1/ Write some ASP/PHP code to skip the authentication requirement for a particular IP address and only index your site from this IP address.

    2/ Write some ASP/PHP code to skip the authentication requirement for a particular user agent. For example the Google user agent name is "Googlebot". Zoom V4 is "ZoomSpider - wrensoft.com"

    3/ Index you site in offline mode from the local file system. There will be no authentication requirement in this case. This is not a good option for database driven or dynamic sites however.

    4/ Have a secret table of contents page to points directly to the content of the site without going through the authentication step. Start Zoom on this secret page.

    We will probably look at other more advanced solutions in a future version of Zoom.

    See also this FAQ for more details. How to index sites requiring authentication with Zoom.

    ------
    David
    Wrensoft
Working...
X