Welcome to WebmasterWorld Guest from 107.20.54.98

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
WebEco/1.0
Anyone know what it is?
2 Kerrin 7:10 am Nov 22, 2002
phpgetter
strange activity from 208.17.76.138
4 PandaM 3:12 am Nov 22, 2002
New one to me Quibot 3.0
3 cmoewes 8:59 pm Nov 21, 2002
NetNoseCrawler/v1.0
no robots.txt
2 Finder 8:36 pm Nov 21, 2002
"HTML Text Download Class" Agent
Does not respect robots.txt
3 carfac 4:35 pm Nov 21, 2002
Sneaky Fast Spider
4 Josk 4:33 pm Nov 21, 2002
Anyone know this bot?
216.55.138.108
8 Weblamer 10:10 pm Nov 20, 2002
AV Scooter 1.0
images
16 wilderness 11:35 pm Nov 19, 2002
server1.business2www.com b2w/0.1
9 Brett_Tabke 10:10 am Nov 18, 2002
Scooter 3.2 IS Ruining My Marriage
Someone tell me what is going on please
9 brina 6:07 pm Nov 17, 2002
Does anyone know anything about these three user agents?
Download Ninja 7.0, Iria/1.07a, and Pockey/4.10.2(Win32; GUI; ix86)
3 GaryK 3:12 pm Nov 17, 2002
webcollage/1.87
no robots.txt
7 bull 5:15 pm Nov 16, 2002
IBM Crawl_Application
198.81.209.19
2 WebGuerrilla 12:54 am Nov 16, 2002
Links 2.1pre3
legitimate browser
2 Finder 9:41 pm Nov 15, 2002
curl/7.7.1
no robots. grabbed my odp dir
7 bull 11:05 pm Nov 13, 2002
New Spam bot?
7 willtell 2:24 am Nov 13, 2002
What in the world is this?
someone requesting www.intel.com and got 200 code.....
12 kstprod 9:25 pm Nov 12, 2002
parallelContextFocusCrawler1.1
not a good guy
2 carfac 8:57 am Nov 12, 2002
Help with an IP
6 carfac 5:48 pm Nov 11, 2002
ebay robot hitting hard!
Why?
22 Liane 4:32 pm Nov 11, 2002
attempted to access form mail
8 Busynut 12:17 am Nov 9, 2002
PHP/4.1.2 user agent
What's this?
3 Scooter24 10:41 am Nov 8, 2002
Was this the Googlebot?
Or was it Someone from Google Surfing Our Site?
10 The_Toecutter 7:33 am Nov 8, 2002
TOPIC/1.0+libwww/5.2.8
What is this?
3 QNetwork 10:02 pm Nov 7, 2002
avfw.sv.av.com
What's AltaVista up to?
6 Key_Master 8:55 pm Nov 7, 2002