Forum Moderators: open

Message Too Old, No Replies

Dear Yahoo.

Please use a real useragent.

         

bobothecat

12:40 am on Jan 7, 2006 (gmt 0)



If you're going to crawl my site, could you please use a 'real' user-agent?

59.11.248.33 - - [06/Jan/2006:18:18:56 -0600] "GET / HTTP/1.0" 403 198 "-" "-"
66.228.182.183 - - [06/Jan/2006:18:19:00 -0600] "GET / HTTP/1.1" 403 230 "-" "-"
66.228.182.187 - - [06/Jan/2006:18:19:00 -0600] "GET / HTTP/1.1" 403 229 "-" "-"
66.228.182.182 - - [06/Jan/2006:18:22:39 -0600] "GET / HTTP/1.1" 403 230 "-" "-"
66.228.182.186 - - [06/Jan/2006:18:23:11 -0600] "GET / HTTP/1.1" 403 230 "-" "-"
210.123.219.48 - - [06/Jan/2006:18:24:25 -0600] "GET / HTTP/1.0" 403 198 "-" "-"
66.228.182.182 - - [06/Jan/2006:18:27:52 -0600] "GET / HTTP/1.1" 403 230 "-" "-"
66.228.182.182 - - [06/Jan/2006:18:27:52 -0600] "GET / HTTP/1.1" 403 229 "-" "-"

jcoronella

4:52 am on Jan 8, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Two of those don't reverse to Yahoo for me.

rlx-2-2-2.labs.corp.yahoo.com

Could be just employees, and not their crawl infrastructure.

bobothecat

9:40 pm on Jan 13, 2006 (gmt 0)



Could be just employees, and not their crawl infrastructure.

Whoops... we can discount the 210. and 59. ( so much for me proof-reading :) ).

I doubt it's an employee ( unless he/she is really bored ) since the same occurs on about 10 sites of mine, all using no UA for the past month now.