Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| baidu Transcoder |
5 | Pfui | 8:23 pm Aug 14, 2011 | ||||
| SISTRIX Crawler sistrix.com/net/de SEO ignores robots.txt |
2 | Pfui | 8:53 pm Aug 9, 2011 | ||||
| static.cloud-ips |
4 | keyplyr | 7:49 am Aug 8, 2011 | ||||
| Hurricane Electric. Bot? Good or bad? |
7 | Bewenched | 1:54 am Aug 7, 2011 | ||||
| Windows-Live-Social-Object-Extractor-Engine |
5 | dstiles | 10:03 pm Aug 6, 2011 | ||||
| Bergdorf Group |
7 | keyplyr | 8:47 pm Aug 5, 2011 | ||||
| Block Search Engines From Indexing Member Profiles |
10 | anthonyon | 7:48 pm July 30, 2011 | ||||
| Google's Fox Offers to Guard Your Henhouse Whither thou goest, local access controls? |
2 | Pfui | 4:11 pm July 29, 2011 | ||||
| where's the browser? |
19 | lucy24 | 7:36 pm July 27, 2011 | ||||
| Bender the web crawler anyone know what this new googlebot is about? |
12 | Kelowna | 1:57 pm July 26, 2011 | ||||
| UAs that asked for robots.txt |
11 | incrediBILL | 6:02 am July 18, 2011 | ||||
| Yellow Pages bot impersonating Googlebot? |
11 | mrtonyg | 9:57 pm July 15, 2011 | ||||
| m160.cs.uwaterloo.ca block? |
4 | santapaws | 7:40 pm July 15, 2011 | ||||
| evrinid |
6 | Pfui | 8:37 pm July 13, 2011 | ||||
| Garlik? A UK outfit |
5 | tangor | 2:52 am July 13, 2011 | ||||
| back in the Ukraine if the User Agent fits, wear it |
23 | lucy24 | 8:22 pm July 9, 2011 | ||||
| a legitimate blank? |
26 | lucy24 | 3:52 pm July 9, 2011 | ||||
| Re-visiting Data URI Unexpected data:image URI calls to web sites |
6 | dstiles | 9:24 pm July 8, 2011 | ||||
| the faviconbot's new clothes |
3 | lucy24 | 11:45 pm July 7, 2011 | ||||
| SiteSpeedBot and misinformation, bad bot |
5 | incrediBILL | 5:22 am July 2, 2011 | ||||
| q1 |
2 | Pfui | 10:03 pm June 29, 2011 | ||||
| where old browsers go to die Netscape hits bottom |
2 | lucy24 | 9:53 am June 28, 2011 | ||||
| User agent chomping bandwidth Syntax error? |
8 | didibreakit | 9:30 pm June 26, 2011 | ||||
| Is it Googlebot or a user coming from Google search? |
4 | jojy | 1:02 pm June 25, 2011 | ||||
| WordPad and htaccess |
4 | wilderness | 10:04 pm June 24, 2011 |
