Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Fetch API Request Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0) Fetch API Request |
5 | jamesyap | 4:27 am Feb 28, 2003 | ||||
| Who is this? I have never seen thi s one |
2 | Laurel | 1:08 am Feb 28, 2003 | ||||
| WebcraftBoot Anyone shed some light on this lil' critter? |
3 | ritch_b | 4:07 pm Feb 27, 2003 | ||||
| Please help with .htaccess ban 163.179.0.0 to 163.179.255.255 range |
10 | nativenewyorker | 5:53 am Feb 27, 2003 | ||||
| Under protecting in IP ranges being tolerant and safe NEVER Pays |
6 | wilderness | 2:58 am Feb 27, 2003 | ||||
| 212.20.147.34 -> Whois? |
2 | akvario | 8:42 pm Feb 26, 2003 | ||||
| 66.207.121.61, P-P-C Cancel, {World Online}, pide-09-00 Violated robots.txt, changed U-A four times in two minutes |
4 | jazzguy | 6:08 pm Feb 26, 2003 | ||||
| Microsoft+URL+Control+-+6.00.8862 What is this fella up to...? |
4 | ukgimp | 3:25 pm Feb 26, 2003 | ||||
| page rank and google crawler |
8 | Louise | 11:05 am Feb 26, 2003 | ||||
| FAST-WebCrawler/3.2 test and Fast Crawler v 3.3 (beta) What's the difference? |
2 | pendanticist | 4:39 am Feb 26, 2003 | ||||
| Topaktuelle Nachrichten Anyone Know Who/What This Is? |
4 | pendanticist | 4:39 am Feb 26, 2003 | ||||
| Yell Spider Is this coming from yell.com? |
2 | IanTurner | 10:38 pm Feb 25, 2003 | ||||
| The use of Includes is it recommended |
2 | fashezee | 3:21 pm Feb 25, 2003 | ||||
| Confuzzledbot/2.0 (+BETA http://bot.confuzzled.lu/) What is it? |
2 | GaryK | 1:09 am Feb 25, 2003 | ||||
| Heavy traffic from IP, ... ... does anyone know? |
6 | Guenni | 8:56 pm Feb 24, 2003 | ||||
| lobo.yahoo.com |
3 | leoo24 | 8:03 pm Feb 24, 2003 | ||||
| IP Query |
4 | StanTheMan | 11:03 am Feb 24, 2003 | ||||
| NetNoseCrawler/v1.0 ...follow up to an old thread more than anything... |
2 | Dreamquick | 11:04 pm Feb 23, 2003 | ||||
| Help needed identifying some kind of automated crawl from 66.122.16. |
8 | bobmark | 8:57 pm Feb 23, 2003 | ||||
| Fast using PycURL UA? Not fetching robots.txt |
4 | jazzguy | 8:04 pm Feb 23, 2003 | ||||
| Bot from netzhase Hit hard last night. |
7 | carfac | 5:37 pm Feb 22, 2003 | ||||
| WebFilter |
4 | wilderness | 5:32 pm Feb 22, 2003 | ||||
| rumours-agent just adding a bit more info |
2 | carfac | 7:00 am Feb 22, 2003 | ||||
| ia_archiver who gets results? Freshbot? Wondering who buys Alexa results... |
5 | xerxes | 9:11 am Feb 21, 2003 | ||||
| Bots "A Standard" Making backbones compliant |
5 | wilderness | 2:16 am Feb 21, 2003 |
