Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Interesting hit from a VPN with python UA ?checkstatus=areyouok |
3 | SumGuy | 10:38 pm Jan 22, 2024 | ||||
| fasthttp bots |
2 | not2easy | 1:15 am Jan 21, 2024 | ||||
| ImagesiftBot appears to be a reverse image search tool |
9 | Bewenched | 6:53 pm Jan 20, 2024 | ||||
| Cloudflare starts hammering me today Actual, or DOS? |
2 | SumGuy | 4:13 pm Jan 18, 2024 | ||||
| Server Farms 2023 Continuing discussion of hosting and data center IP ranges |
13 | not2easy | 1:59 pm Jan 8, 2024 | ||||
| What is the purpose of fetch.tunnel.googlezip.net? I'm seeing more of it |
1 | SumGuy | 2:31 am Jan 7, 2024 | ||||
| NetpeakCheckerBot From a comcast IP? |
1 | SumGuy | 10:57 pm Dec 29, 2023 | ||||
| Strange referer seen today file:///(...)/domains.txt |
1 | SumGuy | 2:26 am Dec 22, 2023 | ||||
| Odd set of 4 UA's seen today Some sort of probe or test I assume |
2 | SumGuy | 12:21 am Dec 12, 2023 | ||||
| Seeing some strange hits today Bytespider for one |
25 | SumGuy | 3:17 pm Dec 11, 2023 | ||||
| Suspicious hits from Macintosh user agents |
3 | ClosedForLunch | 12:44 am Dec 8, 2023 | ||||
| Barkrowler not asking for robots.txt |
1 | Bewenched | 7:20 pm Dec 2, 2023 | ||||
| SeekportBot 65.108.103.105 |
8 | Bewenched | 5:16 pm Dec 2, 2023 | ||||
| G'day from a new member interested in bots/crawlers/spiders |
27 | profshoelace | 12:19 am Nov 26, 2023 | ||||
| Web hits from duckduckgo favicons-bot Includes strange cookie string |
2 | SumGuy | 5:54 am Nov 21, 2023 | ||||
| Two new scrapers (or maybe AI-related bots) |
11 | asterickx | 12:59 am Nov 8, 2023 | ||||
| Seeing Bingbot hits from 20.15.133.x |
3 | SumGuy | 9:20 am Nov 1, 2023 | ||||
| Does a request for /key/ASWD56xxxCSA have some sort of utility? Google and Yandex make requests for them |
1 | SumGuy | 11:05 pm Oct 28, 2023 | ||||
| DuckDuckBot (Am I hopelessly late to this party?) |
13 | Pfui | 9:05 pm Oct 28, 2023 | ||||
| Slow Scrape Plus images |
10 | blend27 | 10:17 am Oct 28, 2023 | ||||
| AntBot |
1 | lucy24 | 6:11 pm Oct 22, 2023 | ||||
| Zaldamo Search Bot Is this a known bot? |
2 | SumGuy | 5:23 pm Oct 20, 2023 | ||||
| Hammered last night by Mozilla/5.0 (Windows NT 10.0; Win64; x64) Apple bad amazon bot? |
19 | Greasemonkey | 5:22 pm Oct 20, 2023 | ||||
| what is colab.research.google.com ? was a refer for a recent web hit |
1 | SumGuy | 2:02 pm Oct 20, 2023 | ||||
| MicrosoftPreview robots.txt |
15 | dstiles | 5:27 pm Oct 9, 2023 |
