Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| MicrosoftPreview A new-ish bot |
1 | dstiles | 8:36 am Sept 13, 2022 | ||||
| What 3'rd party trust / security / AV-AM do you accept or block? And are you sure you know why they're hitting you? |
2 | SumGuy | 8:15 am Sept 13, 2022 | ||||
| User-agent: Mozlila (sic) Typo or attempt to look legit? |
5 | Martin_Potter | 11:29 pm Sept 12, 2022 | ||||
| robot clusters |
19 | lucy24 | 9:46 am Sept 4, 2022 | ||||
| User-agent: Scrapy |
12 | rb77 | 10:39 am Aug 7, 2022 | ||||
| Bot user-agent: HTTP or HTTPS Things change |
2 | dstiles | 3:10 pm Aug 3, 2022 | ||||
| facebook wider crawls |
8 | lucy24 | 11:50 pm July 25, 2022 | ||||
| Bot blocking and advertising/header bidding |
1 | Ralph_Slate | 8:28 pm July 25, 2022 | ||||
| Spammers Using IPv6 Now? Spammers on Blocked IPv4 IPs Getting Through |
1 | martinibuster | 6:51 pm July 23, 2022 | ||||
| hacking probes from hacked IPs |
9 | dstiles | 2:00 pm July 12, 2022 | ||||
| Anybody Else Getting Bot Attacked Via MSFT IPs? 20.192.0.0/10, 20.33.0.0/16 and More |
6 | martinibuster | 4:53 am June 29, 2022 | ||||
| Paracrawl |
2 | lucy24 | 7:24 pm June 26, 2022 | ||||
| Why Do Bots Generally Only Visit My Navigation? |
7 | RedBar | 3:35 pm June 25, 2022 | ||||
| Bytedance / bytespider IP addresses here are a few of them |
3 | SumGuy | 2:07 am June 20, 2022 | ||||
| A curious set of php file requests today From an IP that should not have made it... |
9 | SumGuy | 12:13 pm June 18, 2022 | ||||
| OVH Hosted Bots & Scrapers |
7 | JamieW | 4:53 pm June 11, 2022 | ||||
| IP blocking lists - Ipxo Fresh source of ip;s to block |
2 | SumGuy | 10:33 am June 8, 2022 | ||||
| Ludicrously stupid form spam |
10 | dstiles | 7:52 am June 8, 2022 | ||||
| Honor or reject HTTP OPTIONS query? (From a suspicious British Telecom /24...) |
3 | SumGuy | 8:31 am May 31, 2022 | ||||
| Blocking Google's Proxies |
20 | dstiles | 8:05 am May 22, 2022 | ||||
| novice IP blacklist question query of the nature of banning IP's vs. IP groups |
5 | Dancing_Otter | 12:52 am May 8, 2022 | ||||
| Blocking web site access using Sec-Fetch |
16 | dstiles | 3:08 pm May 3, 2022 | ||||
| Hetzner Hosted Bots & Scrapers |
3 | JamieW | 8:34 am Apr 30, 2022 | ||||
| User Agent Change For Bing Crawler Bingbot |
3 | engine | 10:15 pm Apr 29, 2022 | ||||
| Brave Search User Agent |
5 | brotherhood_of_LAN | 1:08 am Mar 26, 2022 |
