Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Slurp Gone Wild Took only disallowed files |
2 | GaryK | 9:43 am July 15, 2009 | ||||
| pythumbnail.py taking screenshots? |
3 | GaryK | 9:42 am July 15, 2009 | ||||
| Qryos |
5 | keyplyr | 9:36 am July 15, 2009 | ||||
| Log spam or crawler? mtrack.nl |
7 | GaryK | 6:40 am July 15, 2009 | ||||
| Fake Googlebots - why are they here? Asking about why there are so many fake Googlebots accessing our site. |
8 | wizboy | 3:12 am July 15, 2009 | ||||
| User-Agent "GbPlugin" Annoying HTTP requests for a non-existent .SWF file |
2 | jaynl | 8:44 pm July 14, 2009 | ||||
| Visited by http://tools.geek-tools.org Apparent log spam |
5 | GaryK | 4:49 am July 14, 2009 | ||||
| /(null) |
22 | Pfui | 4:12 pm July 13, 2009 | ||||
| New MSN bot? adidxbot |
12 | dstiles | 6:54 pm July 6, 2009 | ||||
| ovale.ru's: "ovalebot3.ovale.ru facepage" and "search new sites" 1 host, 2 bots, 0 robots.txt |
2 | Pfui | 6:18 pm July 5, 2009 | ||||
| solfo-linkchecker If it looks like a bot... |
2 | Pfui | 6:57 pm July 3, 2009 | ||||
| Tagoobot/3.0 http://www.tagoo.ru |
4 | tpeacock | 1:48 am July 1, 2009 | ||||
| "slurp, yahoo! slurp, slurp/2.0, inktomi slurp, slurp.so/1.0" Is someone at Yahoo playing some sort of joke? |
2 | Pfui | 10:41 pm June 30, 2009 | ||||
| ShrinkTheWeb.com Crawler |
2 | Pfui | 8:44 pm June 30, 2009 | ||||
| Rexyobot |
10 | Pfui | 8:41 am June 30, 2009 | ||||
| WideCircles Analysis Agent |
2 | Pfui | 12:37 am June 30, 2009 | ||||
| Spider Hunting or Site Building? Goes in waves, do you trust your bot blocker? |
8 | incrediBILL | 9:26 pm June 28, 2009 | ||||
| smart.apnoti.com Robot/v1.34 |
3 | keyplyr | 5:53 pm June 28, 2009 | ||||
| DoubleVerify Crawler |
4 | keyplyr | 9:01 am June 26, 2009 | ||||
| Pinboard (pinboard.cantbedone.org) |
4 | Pfui | 3:34 am June 26, 2009 | ||||
| HolmesBot (http://holmes.ge) |
6 | Pfui | 4:20 pm June 23, 2009 | ||||
| Zanran ZanranCrawler/0.2 (helpdesk@zanran.com) |
3 | Frank_Rizzo | 8:22 am June 22, 2009 | ||||
| WordPress support page bot? |
5 | GaryK | 4:19 am June 22, 2009 | ||||
| msnbot using IE UA should i suspect screenshots? |
3 | GaryK | 11:21 pm June 21, 2009 | ||||
| acont.de's ACONTBOT |
15 | Pfui | 11:04 pm June 21, 2009 |
