Forum Moderators: open

Message Too Old, No Replies

AV international - the spider puzzle

No Scooter visit, but pages indexed

         

starec

10:07 am on Mar 5, 2001 (gmt 0)

10+ Year Member



Scooter just keeps ignoring me, but all of sudden es.altavista.com has started to bring traffic to my site. All my pages are indexed there. So, the puzzle is, what indexed me? Apparently, there are more AV indexes as the pages do not appear in the normal AV (altavista.com) index. But do local AV versions also use different spiders? Anybody can bring some light into this issues? (I did not see any related discussion the Europen SE forum)

rogerd

1:53 pm on Mar 5, 2001 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



I have had a similar experience. A couple of old doorway pages in a directory off-limits to Scooter and Mercator have appeared in the database. I looked back through the logs to see if I could detect when they were visited, but no luck...

traveller

11:58 am on Mar 6, 2001 (gmt 0)

10+ Year Member



Mercator-1.1 and Mercator-1.2 seems to be the user agents most commonly used by the international sites. As far as I know, Scooter is not used for the international crawls, nor does it seem to be used for any other crawls either (Scooter-1.0 is just a Mercator cloaking, the real Scooter is Scooter/1.0).

starec

7:28 pm on Mar 6, 2001 (gmt 0)

10+ Year Member



Thanks, that's it, Mercator is there in the logs. I saw him before but I thought they do not use Mercator for real spidering, I read somewhere here that they are just testing it...