Welcome to WebmasterWorld Guest from 54.160.198.60

Forum Moderators: goodroi

Message Too Old, No Replies

Ask Jeeves robots.txt entry

what UA should be used

     
4:10 pm on Nov 7, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Oct 1, 2002
posts:96
votes: 0


Mozilla/2.0 (compatible; Ask Jeeves)

Hi Guys, I'm a little confused as to what I should call the Ask Jeeves/teoma/directhit crawler in my robots.txt file. What should I use?

4:19 pm on Nov 7, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


Kerrin,

User-agent: Ask Jeeves

Jim

4:40 pm on Nov 7, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Oct 1, 2002
posts:96
votes: 0


Thanks Jim,

I've seen robots.txt including teoma, directhit and a few varients as UA's for aj.com's crawlers, I just wasn't sure what UA Ask Jeeves actually looks for in robots.txt.