Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| DBLBot dontbuylists.com |
6 | dstiles | 7:14 am Dec 27, 2008 | ||||
| Do Bot-Blocking Techniques Alter Bot Behavior? Various methods discussed, 403 forbidden vs 200 OK[2] ( 1 2 ) |
57 | dstiles | 3:30 am Dec 24, 2008 | ||||
| YoudaoBot |
6 | mcneely | 12:26 pm Dec 23, 2008 | ||||
| P3P Policy |
4 | wilderness | 9:48 pm Dec 22, 2008 | ||||
| Feedfetcher-Google-iGoogleGadgets Something new from Google? |
4 | GaryK | 4:23 pm Dec 21, 2008 | ||||
| Google manual review which user agent do they use? |
5 | SEOPTI | 9:10 am Dec 19, 2008 | ||||
| 127.0.0.1 visits from 127.0.0.1 |
4 | dolcevita | 8:10 am Dec 19, 2008 | ||||
| Yahoo-Test ignoring robots.txt |
3 | dstiles | 6:27 am Dec 18, 2008 | ||||
| "gmail.com" bots? A bunch are showing up |
7 | tangor | 6:16 am Dec 18, 2008 | ||||
| cscinfo.com aipbot bot.com consult dynamics? |
7 | Megaclinium | 5:26 pm Dec 15, 2008 | ||||
| NetcraftSurveyAgent from new Amazon EC2 range didn't read robots.txt |
8 | thetrasher | 4:54 am Dec 15, 2008 | ||||
| Form spammer |
7 | wilderness | 2:20 am Dec 14, 2008 | ||||
| Webcollage Barrage bizarre consecutive distributed hits |
24 | incrediBILL | 3:58 pm Dec 13, 2008 | ||||
| Technorati Bot |
2 | engine | 2:08 pm Dec 10, 2008 | ||||
| Broken MSIE User-Agent nested MSIE strings |
8 | dstiles | 2:00 pm Dec 9, 2008 | ||||
| What is ATT.net/s/s.dll and why is it spamming my website? Is it possibly related to a security overhaul I performed? |
2 | JS_Harris | 6:13 pm Dec 8, 2008 | ||||
| WHttpTest and WinInetSimpleRequest Unwanted and abusive |
6 | GaryK | 4:23 pm Dec 8, 2008 | ||||
| WordTracker Attempts Crawling My Site! Trying to be low key at 180 pages in 14 days |
8 | incrediBILL | 7:03 pm Dec 4, 2008 | ||||
| YesupBot - No soup for you! Bot or real visitor? |
12 | caribguy | 9:10 pm Dec 3, 2008 | ||||
| Fake Googlebot from StandardShell Spoofed UA, no additional headers |
14 | jdMorgan | 11:58 pm Dec 2, 2008 | ||||
| rdfbot drops "Nutch" UA Change |
3 | caribguy | 9:46 am Nov 30, 2008 | ||||
| Identifying bots via ip and host Identifying bots via ip and host |
30 | enigma1 | 5:20 pm Nov 29, 2008 | ||||
| CFNetwork again. More junk from Yahoo? |
7 | GaryK | 7:31 pm Nov 28, 2008 | ||||
| User Agent = Feedfetcher-Google |
17 | smallcompany | 5:02 pm Nov 28, 2008 | ||||
| GurujiBot/1.0 Obeyed Robots.txt |
3 | mcneely | 2:31 am Nov 28, 2008 |
