PDA

View Full Version : / & /index.php both show-up in search results


thewebmaster
12-29-2007, 07:42 AM
I have links to / & /index.php from multiple pages and in search results I have two lines pointing to the same page. I've disabled spidering files w/o extension and used CRC to skip identical content (last one is useless for me because of dynamic ads on pages). Log file said .php indexed 1000, generated setting.php said $NumPages = 1002.
(those 2 I'd like to get rid off)
looks like a BUG to me, any ideas?

wrensoft
12-29-2007, 08:41 PM
I don't think there is any bug. You probably have a start point of /, so that home page gets indexed. Then later on the index.php page also gets found and indexed. The CRC check fails to filter the page becuase your pages are never identical due to ads. The content of the settings file is, I believe, correct based on what you indexed.

I think the simple solution is to either,
1) Only link to 1 home page on your site. i.e. always link to / and never to index.php
OR
2) Start the indexer at index.php rather than / and filter pages without extension
OR
3) Start the indexer at / but then filter the full URL of your index.php home page (using the page and folder skip list).