Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Google spidering with new user agent |
5 | zerillos | 10:01 pm Apr 9, 2011 | ||||
| How to find all the IP ranges of a hosting co |
15 | Megaclinium | 3:15 am Apr 8, 2011 | ||||
| Googlebot with Referer |
5 | dstiles | 10:11 pm Apr 5, 2011 | ||||
| Bingbot Full Trip DNS Validation is BROKEN! |
9 | incrediBILL | 9:31 pm Apr 2, 2011 | ||||
| Microsoft launches Azure |
2 | wilderness | 8:13 pm Apr 1, 2011 | ||||
| Bork Edition Not the Swedish chef |
2 | caribguy | 9:35 pm Mar 29, 2011 | ||||
| Spider / Bit / Agent rippers 0 in stats |
2 | jim_knopf | 2:14 am Mar 29, 2011 | ||||
| LeaseWeb? |
6 | Bewenched | 10:06 pm Mar 28, 2011 | ||||
| What is Bing's / Yahoo's mobile bot user-agent string? |
4 | jersey_guy | 1:58 pm Mar 24, 2011 | ||||
| Visitor using Yahoo proxy. Why? User Agent string remains the same. |
12 | grandma_genie | 3:32 am Mar 24, 2011 | ||||
| Microsoft Office user agent type of the agent |
3 | BarataPT | 8:07 am Mar 20, 2011 | ||||
| Google Wireless Transcoder Is this a cell phone user agent? |
22 | grandma_genie | 5:35 am Mar 10, 2011 | ||||
| As8075 Microsoft |
3 | wilderness | 10:34 pm Mar 9, 2011 | ||||
| Stopping scrapers from the get-go [2] ( 1 2 ) |
46 | wheel | 3:18 am Mar 4, 2011 | ||||
| Internet Archive's Archive-It '[Subscribers] harvest, catalog, manage...' |
8 | Pfui | 1:23 am Mar 3, 2011 | ||||
| GoogleBot from China First time for me, |
5 | blend27 | 11:07 pm Mar 2, 2011 | ||||
| Abuse from a Google IP range |
4 | Mokita | 10:27 pm Feb 20, 2011 | ||||
| a new bot bot |
3 | mojomike | 1:14 am Feb 19, 2011 | ||||
| Dragon135 Suspicious crawler leading back to strange open directory |
8 | Maintainer | 9:09 pm Feb 17, 2011 | ||||
| Windows Live Social Object Extractor ? |
2 | Umbra | 12:34 am Feb 17, 2011 | ||||
| Exalead ? Massive spidering |
2 | Bewenched | 12:27 am Feb 17, 2011 | ||||
| LightSpeedSystems Why Spoof? |
3 | TheMadScientist | 5:18 am Feb 16, 2011 | ||||
| Exabot Thumbnails Block or not to block? |
2 | tangor | 1:07 pm Feb 12, 2011 | ||||
| WordPress |
4 | Umbra | 8:36 pm Feb 11, 2011 | ||||
| What is the ADM token for? |
4 | GaryK | 9:24 pm Feb 10, 2011 |
