Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| How To Block trek6.sv.av.com from my site trek6.sv.av.com takes up too much bandwidth |
6 | akogo | 8:59 pm Aug 2, 2003 | ||||
| Exalead NG/MimeLive Client (convert/http/0.123) New bot or former NG/1.0? |
2 | claus | 10:21 pm Aug 1, 2003 | ||||
| WebSearchBench no robots either |
2 | bull | 6:30 pm Aug 1, 2003 | ||||
| Zibie Spider 0.1 Java/1.4.2 no robots. |
2 | bull | 5:38 pm Aug 1, 2003 | ||||
| Good guy spiders and the Bad ones! Is there a list |
7 | atlinx | 3:50 pm Aug 1, 2003 | ||||
| My list is outdated |
2 | rainborick | 2:21 pm Aug 1, 2003 | ||||
| Ano-Kato Greek SE |
3 | volatilegx | 2:18 pm Aug 1, 2003 | ||||
| zeus 41852 webster pro v2.9 win32? zeus 41852 webster pro v2.9 win32 bot |
4 | mcavill | 10:11 am Aug 1, 2003 | ||||
| DigExt |
4 | Tonearm | 12:35 am Aug 1, 2003 | ||||
| QuePasaCreep v0.9.13 from ipa 69.28.130.229 new semi-bogus spider? |
18 | rodent | 9:01 am July 31, 2003 | ||||
| Anyone recognise this? |
2 | davelms | 7:40 pm July 30, 2003 | ||||
| New Exalead UA string? |
3 | jrobbio | 12:44 am July 30, 2003 | ||||
| Spoofing LookSmart? Fell into Spider Trap |
2 | frontpage | 11:50 pm July 29, 2003 | ||||
| Anybody recognise this ip? Just checking the log's when I found this |
13 | Glovebox | 10:55 pm July 29, 2003 | ||||
| allowing 403 page to denied IPs |
5 | nancyb | 10:40 pm July 29, 2003 | ||||
| AOLserver |
2 | wilderness | 4:40 pm July 29, 2003 | ||||
| Dolly/1.0 Anyone else see this? |
8 | Finder | 9:41 pm July 28, 2003 | ||||
| 216.88.0.0-216.91.255.255 Savvis Communications - why spidering? |
6 | trillianjedi | 5:55 pm July 28, 2003 | ||||
| Another one banned |
7 | cyberkat | 2:40 pm July 28, 2003 | ||||
| Typical spider activity beginner's question |
2 | Mr_Busby | 3:26 am July 28, 2003 | ||||
| WebTV and Verio |
5 | wilderness | 4:10 am July 27, 2003 | ||||
| why are spiders bad? sorry if this is a rookie question |
12 | SammiGirl | 12:33 am July 27, 2003 | ||||
| Fake GoogleBot? Korean NaverRobot bot masking as "GoogleBot" |
6 | Romeo | 3:57 pm July 26, 2003 | ||||
| Fake Dynamic IP changing continusly Robot with Dynamic changing IP |
3 | nilloc | 3:13 pm July 26, 2003 | ||||
| Any ideas: arirang_check What is it? |
3 | animator | 5:13 pm July 25, 2003 |
