Forum Moderators: open

Message Too Old, No Replies

most common robots user-agent

         

kelly_boyce

12:01 pm on Aug 18, 2003 (gmt 0)

10+ Year Member



Hello,

I have a cloaking script that displays two different versions of my site depending on the user-agent. Can anyone plz tell me what are the most common spider user-agents? e.g. yahoo, google etc..

thanx

mack

12:03 pm on Aug 18, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Google = Googlebot
Altavista = Scooter
Inktomi = Slurp

Hope this helps.

kelly_boyce

12:05 pm on Aug 18, 2003 (gmt 0)

10+ Year Member



any other common ones?

mack

12:12 pm on Aug 18, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



[robotstxt.org...]

One or two more here ;-)

Mack.

lindajames

12:27 pm on Aug 18, 2003 (gmt 0)

10+ Year Member



what spider is the user-agent "ODP" for?

mack

12:34 pm on Aug 18, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



I dont think there is a spider with that user agent.

You may be thinking along the lines of "Robozilla". This is a sider used by the Open Directry Project (odp) for finding dead links within the directory.

Mack.

Friday

5:24 pm on Aug 19, 2003 (gmt 0)

10+ Year Member



NEVER, EVER cloak by user_agent only. It's too easy for your competitors to mimic a user_agent and see exactly what you're doing and report you.

If you're going to cloak, cloak by IP address.
It's a pain, to be sure, keeping an up-to-date IP list, but getting "unbanned" by the SEs and then trying to climb back to the top of the SERPs is a MUCH bigger pain.

Cheers,
Friday