Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Can spiders crawl members only pages? |
5 | AjiNIMC | 9:29 pm Oct 15, 2003 | ||||
| Mozilla/4.0 hhjhj@yahoo.com it's back w/new IP |
3 | coyote | 6:54 pm Oct 15, 2003 | ||||
| 80.177.2.139 - MSIECrawler Is this legit? |
3 | rooshine | 8:00 am Oct 15, 2003 | ||||
| Space Bison/0.02 [fu] (Win67; X; SK) What's this beast |
4 | seindal | 11:45 pm Oct 14, 2003 | ||||
| How do i make a simple bot trap |
3 | hafnius | 11:29 pm Oct 14, 2003 | ||||
| Bot hitting alot |
13 | Rivethed | 3:35 am Oct 14, 2003 | ||||
| Help explain unusual GET requests from spider/downloader GET /' and GET /MSOffice/cltreq.asp? |
10 | Wizcrafts | 1:22 am Oct 14, 2003 | ||||
| New bot came to see me -- WebSurvey Java ? |
2 | guillermo5000 | 1:08 am Oct 14, 2003 | ||||
| semanticdiscovery/0.2 has problems with robots.txt...or a forged UA |
3 | WebJoe | 8:04 pm Oct 13, 2003 | ||||
| Mozilla/5.0 (Version: XXXX Type:XXXX) |
2 | BlueSky | 4:32 pm Oct 12, 2003 | ||||
| InetURL:/1.0 Doesn't read robots.txt ... |
4 | bobothecat | 10:48 pm Oct 11, 2003 | ||||
| Brief UA's |
6 | wilderness | 3:43 pm Oct 11, 2003 | ||||
| What do you watch the spiders with? Any good tools? |
6 | stuntdubl | 2:24 pm Oct 11, 2003 | ||||
| puf/0.91beta6a (Linux 2.4.19-metacarta-appliance-2; i686) Savagely attacked a site of mine |
3 | seindal | 6:02 pm Oct 9, 2003 | ||||
| 207.213.165.104 204K hits |
2 | hanuman | 11:10 pm Oct 8, 2003 | ||||
| googlebot ips |
6 | anand | 10:07 pm Oct 8, 2003 | ||||
| conducent ip? |
5 | coffeebean | 9:29 am Oct 8, 2003 | ||||
| Detecting spoofed user agents Best method? |
3 | Demeter | 7:09 am Oct 8, 2003 | ||||
| AHHH help |
3 | ganderla | 7:39 pm Oct 7, 2003 | ||||
| Never seen this from Google before... What the heck is going on? |
3 | guillermo5000 | 5:14 am Oct 6, 2003 | ||||
| Fake AV log spammer? |
3 | wilderness | 9:41 pm Oct 5, 2003 | ||||
| PersonaPilot/1.00 |
2 | BergtheRed | 12:15 pm Oct 5, 2003 | ||||
| Tide Is this a new msn spider? |
9 | quiet | 5:10 am Oct 5, 2003 | ||||
| Wavepluz what is this? |
9 | coyote | 4:17 am Oct 5, 2003 | ||||
| possible scanning agent |
2 | wkitty42 | 1:37 am Oct 4, 2003 |
