Welcome to WebmasterWorld Guest from 220.127.116.11
PreModerated forum to id search engine spiders
|Thread Subject||Messages||Started by||Last Message|
robots.txt, robotpack, generator
|2||DarkShadow||4:09 am Jan 13, 2003|
|Production Bot DOT 3216B
Anyone come across this one?
|8||acee||12:30 am Jan 13, 2003|
|18||fashezee||2:05 am Jan 12, 2003|
|Strange User agent
||2||Orange_XL||11:53 pm Jan 8, 2003|
|perl tool for highlighting googlebot
||2||ga_ga||7:51 am Jan 8, 2003|
|New Apple Browser Safari User Agent
Not a bot.
|2||transistor||11:22 pm Jan 7, 2003|
||8||Orange_XL||12:07 pm Jan 7, 2003|
|What is this Bot?
|4||Rugles||4:25 pm Jan 6, 2003|
||7||yasunglass||12:24 pm Jan 6, 2003|
Who/what is this?
|11||mayor||3:02 pm Jan 5, 2003|
|Willow Internet Crawler
Anyone have experience of this one?
|3||mack||5:32 pm Jan 3, 2003|
|Ziggy -- The Clown From Hell
So who is this?
|2||Powdork||7:21 am Jan 3, 2003|
|What's the relation between google and alexa
who knows what is the relation between google and alexa, thank!
|4||hotice_2002||5:47 am Jan 3, 2003|
|1||tuna16||11:33 pm Jan 1, 2003|
who is this?
|2||HuhuFruFru||8:11 am Dec 29, 2002|
|Blocking Bad Bots.
This is not working for me.
|8||Elgoog||6:17 am Dec 29, 2002|
|Spider from multiple IP's all at once
Spider from multiple IP's all at once
|5||sammy||6:13 am Dec 29, 2002|
|I need help building a list
I am building a list of spiders that we should let in and would like help
|6||anoryu||6:10 am Dec 29, 2002|
|Teradex Mapper violating robots.txt
in fact, they never bothered to check
|2||Finder||6:08 am Dec 29, 2002|
|Fast PartnerSite Crawler Question
||4||Jon_King||2:49 pm Dec 24, 2002|
|Microsoft URL Control
A warning and a request for information
|29||Crazy_Fool||12:45 am Dec 23, 2002|
|Universal Payment Gateway
|4||wilderness||7:50 pm Dec 22, 2002|
|1||wilderness||4:52 am Dec 22, 2002|
|New Spam Bot?
Hit by "Lite Bot 0816b"
|5||121focus||10:03 am Dec 21, 2002|
didn't grab much, but....
|1||carfac||5:52 pm Dec 20, 2002|