Welcome to WebmasterWorld Guest from 54.205.60.171

Forum Moderated by: Ocean10000 & incrediBILL & keyplyr

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
webcollage/1.87
no robots.txt
7 bull 5:15 pm Nov 16, 2002
IBM Crawl_Application
198.81.209.19
2 WebGuerrilla 12:54 am Nov 16, 2002
Links 2.1pre3
legitimate browser
2 Finder 9:41 pm Nov 15, 2002
curl/7.7.1
no robots. grabbed my odp dir
7 bull 11:05 pm Nov 13, 2002
New Spam bot?
7 willtell 2:24 am Nov 13, 2002
What in the world is this?
someone requesting www.intel.com and got 200 code.....
12 kstprod 9:25 pm Nov 12, 2002
parallelContextFocusCrawler1.1
not a good guy
2 carfac 8:57 am Nov 12, 2002
Help with an IP
6 carfac 5:48 pm Nov 11, 2002
ebay robot hitting hard!
Why?
22 Liane 4:32 pm Nov 11, 2002
attempted to access form mail
8 Busynut 12:17 am Nov 9, 2002
PHP/4.1.2 user agent
What's this?
3 Scooter24 10:41 am Nov 8, 2002
Was this the Googlebot?
Or was it Someone from Google Surfing Our Site?
10 The_Toecutter 7:33 am Nov 8, 2002
TOPIC/1.0+libwww/5.2.8
What is this?
3 QNetwork 10:02 pm Nov 7, 2002
avfw.sv.av.com
What's AltaVista up to?
6 Key_Master 8:55 pm Nov 7, 2002
Adwords Spider
(and no -- its not from Google)
2 Josk 10:23 am Nov 7, 2002
Pop.gr/6.1
ip: 195.186.149.91
2 PandaM 9:54 am Nov 7, 2002
rico/0.1
11 PandaM 4:05 am Nov 7, 2002
Perlscript Downloading Images
12-228-84-214.client.attbi.com; libwww-perl/5.65
4 weesnich 9:59 am Nov 6, 2002
Full Web Bot 0516B
no robots.txt, private bot
4 bull 4:16 pm Nov 5, 2002
Does anyone know IPiumBot?
12 casualsub 8:00 pm Nov 4, 2002
NetNewsWire
2 wilderness 7:12 pm Nov 4, 2002
New Walhello Spider
Well, hello there little spider...
6 Josk 9:21 pm Nov 2, 2002
kototoi - zao
sounds friendly
3 john316 4:57 pm Nov 2, 2002
UA: THTTP: HTTP API Test Application
1 carfac 4:52 pm Nov 2, 2002
Netscape Spider? weird entry in logs....
2 kstprod 3:43 pm Nov 1, 2002

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members