Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| UA: XML Sitemaps Generator |
2 | tangor | 2:42 pm May 24, 2020 | ||||
| AbuseIPDB down |
4 | LifeinAsia | 5:15 pm May 19, 2020 | ||||
| today's snicker |
17 | lucy24 | 4:42 pm May 19, 2020 | ||||
| checkip.amazonaws.com/ HTTP/1.1 |
3 | tangor | 4:48 pm May 9, 2020 | ||||
| robots and https things I’ve learned |
16 | lucy24 | 10:35 pm Apr 30, 2020 | ||||
| SwissCows secure, private search engine |
9 | dstiles | 7:10 pm Apr 17, 2020 | ||||
| Clarabot What's heppened to it? |
6 | dstiles | 3:45 pm Apr 7, 2020 | ||||
| serpstatbot |
2 | lucy24 | 10:12 pm Apr 5, 2020 | ||||
| msn & js |
13 | wilderness | 6:55 pm Mar 31, 2020 | ||||
| okhttp/3.8.1 |
3 | tangor | 12:31 am Mar 30, 2020 | ||||
| another day, another botnet |
5 | lucy24 | 5:11 pm Mar 21, 2020 | ||||
| DomainStatsBot He is back. |
6 | Martin_Potter | 12:35 am Mar 12, 2020 | ||||
| Don't shoot the hostages - sparing DuckDuckGo on AWS Poking holes in AWS ranges |
16 | JamesSC | 8:58 am Mar 8, 2020 | ||||
| What are these +1200 USA datacenters bots doing to my site ? |
24 | adrianTNT | 4:02 pm Feb 26, 2020 | ||||
| node-superagent/3.8.3 |
13 | hekasi | 10:51 am Feb 26, 2020 | ||||
| reqwest/0.9.19 All it wants is images |
3 | tangor | 11:17 pm Feb 20, 2020 | ||||
| Not sure but the wall works for now.. |
9 | blend27 | 4:01 pm Feb 15, 2020 | ||||
| msnbot still active? |
6 | JorgeV | 12:40 pm Feb 10, 2020 | ||||
| Yandexbot |
3 | JorgeV | 12:29 pm Feb 10, 2020 | ||||
| Some resources for some. |
6 | Dimitri | 11:47 am Feb 10, 2020 | ||||
| Marketing related robots? |
3 | JorgeV | 11:37 am Feb 10, 2020 | ||||
| Thinking of blocking 40.64.0.0/10 and 51.140.0.0/14 (MSFT) Seen some bot stuff recently |
5 | SumGuy | 8:01 pm Feb 4, 2020 | ||||
| Anchorage crawler |
3 | ClosedForLunch | 6:55 am Feb 4, 2020 | ||||
| Phasing out User Agents What will they look like and how will they affect us |
15 | dstiles | 12:19 pm Jan 23, 2020 | ||||
| referer fbapp://nnnnnnnnnnnnn/unknown |
8 | tangor | 1:37 am Jan 19, 2020 |
