Welcome to WebmasterWorld Guest from 54.197.116.116

Forum Moderators: martinibuster

Message Too Old, No Replies

Yahoo spider

Anyone else seen this?

   
2:41 am on Jan 18, 2001 (gmt 0)

10+ Year Member



This little sucker just pops in every couple of weeks and retrieves index.html from one of my sites. No other files - just index.html. Like a link checker.

2001-01-17 16:15:12 wr13.yahoo.com - GET /index.html Mozilla/4.0,+-,+-,+- 200 0 3338 69 1453 HTTP/1.0 Mozilla/4.0 - -

This site is not listed in Yahoo but we have applied 4-5 times before noting that their policy requires sites to be viewed without any plugins. This site uses Flash and states you must have Flash installed to proceed beyond the home page.

In discussions with someone I know from another SEO company I have been told that Yahoo does not use editors to review/select sites for inclusion, it spiders with a Mozilla agent using predefined criteria that browses using a more "natural" behaviour. Said info was supposed to come from someone they knew inside Yahoo.

All hearsay of course but this little beggar is annoying me. Anyone else care to shed some light on this?

- Fusioneer

4:22 am on Jan 18, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>In discussions with someone I know from another SEO company I have been told that Yahoo does not use editors to review/select sites for inclusion...

If I understand what you're saying, there are many people who believe they do have editors who "rove" the internet looking for sites. It makes sense to me, as well, that they do, given the "mysterious" appearance of many listings which have not been submitted.

As luck would have it, there is the beginning of a discussion about just that thing in this thread [webmasterworld.com]. :)