Welcome to WebmasterWorld Guest from 54.162.91.86

Forum Moderated by: Ocean10000 & incrediBILL & keyplyr

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
robots.txt generator
robots.txt, robotpack, generator
2 DarkShadow 4:09 am Jan 13, 2003
Production Bot DOT 3216B
Anyone come across this one?
8 acee 12:30 am Jan 13, 2003

18 fashezee 2:05 am Jan 12, 2003
Strange User agent
2 Orange_XL 11:53 pm Jan 8, 2003
perl tool for highlighting googlebot
2 ga_ga 7:51 am Jan 8, 2003
New Apple Browser Safari User Agent
Not a bot.
2 transistor 11:22 pm Jan 7, 2003
Zeus 2.6
8 Orange_XL 12:07 pm Jan 7, 2003
What is this Bot?
SynoBot
4 Rugles 4:25 pm Jan 6, 2003
IP Address
7 yasunglass 12:24 pm Jan 6, 2003
ec009.inktomi.com
Who/what is this?
11 mayor 3:02 pm Jan 5, 2003
Willow Internet Crawler
Anyone have experience of this one?
3 mack 5:32 pm Jan 3, 2003
Ziggy -- The Clown From Hell
So who is this?
2 Powdork 7:21 am Jan 3, 2003
What's the relation between google and alexa
who knows what is the relation between google and alexa, thank!
4 hotice_2002 5:47 am Jan 3, 2003
webtrends keywords.ini
webtrends keywords.ini
1 tuna16 11:33 pm Jan 1, 2003
y134.mun.yahoo.com
who is this?
2 HuhuFruFru 8:11 am Dec 29, 2002
Blocking Bad Bots.
This is not working for me.
8 Elgoog 6:17 am Dec 29, 2002
Spider from multiple IP's all at once
Spider from multiple IP's all at once
5 sammy 6:13 am Dec 29, 2002
I need help building a list
I am building a list of spiders that we should let in and would like help
6 anoryu 6:10 am Dec 29, 2002
Teradex Mapper violating robots.txt
in fact, they never bothered to check
2 Finder 6:08 am Dec 29, 2002
Fast PartnerSite Crawler Question
4 Jon_King 2:49 pm Dec 24, 2002
Microsoft URL Control
A warning and a request for information
29 Crazy_Fool 12:45 am Dec 23, 2002
Universal Payment Gateway
UP.Browser/3.2.9.1-UPG1 UP.Link/3.2
4 wilderness 7:50 pm Dec 22, 2002
Comscore Networks
216.148.246.70
1 wilderness 4:52 am Dec 22, 2002
New Spam Bot?
Hit by "Lite Bot 0816b"
5 121focus 10:03 am Dec 21, 2002
bdcindexer
didn't grab much, but....
1 carfac 5:52 pm Dec 20, 2002

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members