Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| SqwidgeBot |
4 | keyplyr | 12:32 am July 3, 2008 | ||||
| Which stupid bot is asking for . /foldername/google-analytics.com/ga.js' |
20 | g1smd | 11:06 pm July 1, 2008 | ||||
| How to design a good/nice spider? spider design |
25 | vidaj | 11:32 pm June 28, 2008 | ||||
| Yahoo LVLT |
9 | wilderness | 4:11 pm June 28, 2008 | ||||
| AVG Toolbar Glitch May Be Causing Visitor Loss User Agent Flaw Suspected[6] ( 1 2 3 4 5 6 ) |
173 | Umbra | 11:16 pm June 27, 2008 | ||||
| How to block all those sites that collect site info and past them on there site, a little like alexa, but with whois.... |
2 | zeus | 2:10 am June 21, 2008 | ||||
| PRCrawler Emerges From Stealth Mode as Kindsight ISPs to insert their own ads under guise of security software? |
12 | incrediBILL | 2:05 am June 21, 2008 | ||||
| Yahoo under Linux UA |
4 | Staffa | 1:50 am June 21, 2008 | ||||
| v92 .nat.svl.searchme.com spider.? taking bandwith? |
7 | jim_knopf | 7:21 pm June 19, 2008 | ||||
| AutoProxy4 |
3 | wilderness | 10:36 pm June 17, 2008 | ||||
| Mozilla/5.0 (compatible; ScoutJet; http://www.scout jet.com/) Letting it in, or blocking it? |
4 | g1smd | 6:54 pm June 16, 2008 | ||||
| Is Cogentco = Twiceler? they're back again... |
3 | peterg22 | 3:35 pm June 16, 2008 | ||||
| Baidu Block |
6 | outland88 | 9:10 am June 13, 2008 | ||||
| LinkScanner, AVG, Trend Micro, 1813 and SV1 Disambiguation of secretive anti-virus tools |
21 | Samizdata | 2:49 pm June 10, 2008 | ||||
| Suspicious crawling Any ideas what this is? |
2 | Thez | 10:23 am June 10, 2008 | ||||
| amazonaws goes stealth |
2 | incrediBILL | 7:05 pm June 9, 2008 | ||||
| AOL changing IPs Velocity and other traps |
2 | phred | 10:23 pm June 5, 2008 | ||||
| yoofind yoono webcrawler new look |
11 | Hobbs | 4:27 pm June 3, 2008 | ||||
| Trend Micro AV May Be Causing Excess Traffic [2] ( 1 2 ) |
31 | blend27 | 9:25 pm June 2, 2008 | ||||
| Kalooga |
13 | keyplyr | 4:41 pm June 1, 2008 | ||||
| Are all UAs ending in "SV1)" invalid? |
9 | Mokita | 3:01 pm June 1, 2008 | ||||
| Sitemaps.xml Single requests, various IPs |
2 | Umbra | 5:08 pm May 30, 2008 | ||||
| Google IP Requesting Images No user agent for the images |
7 | incrediBILL | 5:36 am May 30, 2008 | ||||
| Separate Google HOST and ADDR run Java/1.6.0 01 No robots.txt no matter what. |
2 | Pfui | 1:53 pm May 28, 2008 | ||||
| Googlebot from Serbia? |
6 | mrjones | 2:59 am May 28, 2008 |
