Welcome to WebmasterWorld Guest from 54.159.26.69

Forum Moderators: goodroi

Message Too Old, No Replies

Ask Jeeves robots.txt entry

what UA should be used

     

Kerrin

4:10 pm on Nov 7, 2002 (gmt 0)

10+ Year Member



Mozilla/2.0 (compatible; Ask Jeeves)

Hi Guys, I'm a little confused as to what I should call the Ask Jeeves/teoma/directhit crawler in my robots.txt file. What should I use?

jdMorgan

4:19 pm on Nov 7, 2002 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Kerrin,

User-agent: Ask Jeeves

Jim

Kerrin

4:40 pm on Nov 7, 2002 (gmt 0)

10+ Year Member



Thanks Jim,

I've seen robots.txt including teoma, directhit and a few varients as UA's for aj.com's crawlers, I just wasn't sure what UA Ask Jeeves actually looks for in robots.txt.

 

Featured Threads

Hot Threads This Week

Hot Threads This Month