Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

New crawler version using cookies?

Yahoo Vertical Crawler

11:43 am on Mar 5, 2004 (gmt 0)

New User

10+ Year Member

joined:Feb 2, 2003
votes: 0


Is this a new crawler?

Yahoo-VerticalCrawler-FormerWebCrawler/3.9 crawler at trd dot overture dot com; [alltheweb.com...]

This crawler uses cookies.

The old FAST-WebCrawler/3.8 is also visiting my site at the same time AFAIK that one doesn't use cookies.

4:59 pm on Mar 19, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 31, 2003
votes: 0

Yep, it looks new to me, and yep, it uses cookies. Unfortunately, it doesn't seem to use robots.txt.