Welcome to WebmasterWorld Guest from 54.196.238.210

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Heads Up!
9 toolman 3:08 pm Jun 5, 2001
inktomi2-lut.server.ntl.com
.. Found this one grabbing only one page.
2 Rumbas 2:28 pm Jun 5, 2001
psbot/0.1
picsearch is currently indexing pictures
8 Son_House 12:32 pm Jun 5, 2001
216.35.112.143 is Fast ?
10 berno 3:19 am Jun 5, 2001
Why run a 404 test?
If they're searching for stolen content?
6 mivox 12:33 am Jun 5, 2001
MIIxpc?
10 mark_roach 12:03 am Jun 5, 2001
New Altavista Behaviour...
libwww-perl/5.46 for av?
5 skirril 11:51 pm Jun 4, 2001
Alexa IP's
2 toolman 7:05 pm Jun 4, 2001
Difference in slurps
1 bobriggs 3:49 pm Jun 1, 2001
question
a page with a frame
1 mimmo 8:19 am May 31, 2001
excite
spider?
3 Ove 8:40 pm May 30, 2001
webcraft by cybercity.fr
3 berno 8:16 pm May 30, 2001
teoma_agent1 ?
host06.teo.ewr.qwest.net
2 skirril 3:21 pm May 30, 2001
Goo?
4 Brett_Tabke 9:17 am May 30, 2001
Hit Hard 876 reqs in 42 min
ip176.danville3.va.pub-ip.psi.net
3 Son_House 2:18 am May 30, 2001
larbin spider@dunnserver.com Sqworm?
5 balsamic 9:23 am May 29, 2001
sodoit.com
2 Vishal 8:13 am May 29, 2001
MultiText/0.1
1 Froggyman 4:59 am May 28, 2001
Bogus referrer?
2 skirril 2:56 pm May 27, 2001
polybot 1.0
8 Froggyman 7:28 am May 27, 2001
unknown POST action
can anyone tell me what this all about?
11 sedir 11:39 am May 26, 2001
Anyone ever seen this...
3 toolman 9:31 pm May 24, 2001
magick.texcpe.cableone.net
Googlebot UA
3 sugarkane 1:24 pm May 24, 2001
qestion
to refuse approch
4 mimmo5 12:29 pm May 24, 2001
Need a spider tracking script
...In Cold Fusion or VBScript
7 Travoli 2:14 pm May 23, 2001

Featured Threads

Hot Threads This Week

Hot Threads This Month