Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Wikki bots |
3 | dstiles | 9:52 am Jan 25, 2022 | ||||
| Sidetrade indexer bot ? |
2 | SumGuy | 6:56 am Jan 25, 2022 | ||||
| Is it Okay to Block AS8075 MICROSOFT I get scraped from non-MSN bots from that range |
5 | martinibuster | 4:26 am Jan 24, 2022 | ||||
| Server Farms July 2021 Continuing discussion of hosting and data center IP ranges |
13 | not2easy | 2:56 am Jan 3, 2022 | ||||
| Adsbot redux |
2 | lucy24 | 4:26 am Dec 31, 2021 | ||||
| Is This a New Hack Method? |
12 | Angonasec | 2:33 pm Dec 19, 2021 | ||||
| Apple bot? Not according to DNS |
5 | dstiles | 9:51 pm Dec 16, 2021 | ||||
| crawl-66-249-66-66.googlebot.com How? |
8 | blend27 | 6:00 pm Nov 27, 2021 | ||||
| Grover/Grover-1.18 (Web Crawler) |
3 | tangor | 2:15 am Oct 29, 2021 | ||||
| Unusual UA . mostly amusing |
2 | tangor | 3:22 pm Oct 25, 2021 | ||||
| Amazonbot |
2 | lucy24 | 9:39 pm Oct 24, 2021 | ||||
| YandexMobileBot |
6 | dstiles | 6:54 pm Oct 24, 2021 | ||||
| Hits to wp-login.php and xmlrpc.php from residential IP space what exactly is behind these hits? |
16 | SumGuy | 4:34 pm Oct 4, 2021 | ||||
| Non-crawling bots Where have they gone? |
7 | dstiles | 6:24 am Oct 1, 2021 | ||||
| Neevabot |
4 | lucy24 | 7:41 am Sep 5, 2021 | ||||
| Is it Okay to Block Google Cloud IP Range? Getting a Lot of non-Google Bots |
5 | martinibuster | 4:52 pm Aug 28, 2021 | ||||
| DotBot |
9 | lucy24 | 6:22 pm Aug 14, 2021 | ||||
| Odd UA acting bad |
3 | tangor | 5:46 pm Aug 13, 2021 | ||||
| Vagabondo / WiseGuys |
5 | dstiles | 9:20 am Aug 11, 2021 | ||||
| Bingbot on 13.66.139.0/24 Seems to have ended? |
6 | SumGuy | 4:34 pm Aug 1, 2021 | ||||
| Screaming Frog only crawls /en/ pages |
7 | jediviper | 6:50 am Jul 26, 2021 | ||||
| Google-AMPHTML bot Undocumented Google crawler? |
7 | Giacomo | 7:36 pm Jul 15, 2021 | ||||
| Sec-Fetch-* headers Are there any modern browsers using them? |
26 | blend27 | 2:47 am Jul 13, 2021 | ||||
| Atomseobot |
3 | lucy24 | 9:13 pm Jul 4, 2021 | ||||
| Cloud Server with Rogue Bot 154.127.52.0/24 |
7 | Bewenched | 3:40 pm Jul 2, 2021 |
