Welcome to WebmasterWorld Guest from 54.162.44.105

Forum Moderators: goodroi

Message Too Old, No Replies

Ask Jeeves robots.txt entry

what UA should be used

     
4:10 pm on Nov 7, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Oct 1, 2002
posts:96
votes: 0


Mozilla/2.0 (compatible; Ask Jeeves)

Hi Guys, I'm a little confused as to what I should call the Ask Jeeves/teoma/directhit crawler in my robots.txt file. What should I use?

4:19 pm on Nov 7, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


Kerrin,

User-agent: Ask Jeeves

Jim

4:40 pm on Nov 7, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Oct 1, 2002
posts:96
votes: 0


Thanks Jim,

I've seen robots.txt including teoma, directhit and a few varients as UA's for aj.com's crawlers, I just wasn't sure what UA Ask Jeeves actually looks for in robots.txt.

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members