Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: martinibuster
b100104.yse.yahoo.net - - [05/Feb/2014:20:26:20 +0000] "GET /robots.txt" 200 2551 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"
Yahoo provide a little bit of this information on their Slurp web page. help.yahoo.com...
Slurp collects content from partner sites
It also accesses pages from sites across the Web to confirm accuracy and improve Yahoo's personalized content for our users.