Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Go-http-client/1.1 |
8 | Pfui | 4:09 pm Jul 23, 2023 | ||||
| outlook.live.com is this an email link? |
5 | Wayder | 12:36 pm Jul 23, 2023 | ||||
| Amazon Adbot |
1 | Peter_S | 11:45 am Jul 23, 2023 | ||||
| Expanse (yada-yada) Scanning from googleusercontent |
4 | Pfui | 5:22 pm Jul 16, 2023 | ||||
| Seeing something called BitSightBot/1.0 Amazon IP but looks to be internet-census |
3 | SumGuy | 3:40 pm Jul 16, 2023 | ||||
| Linespider (quasi update) |
3 | Pfui | 6:13 pm Jul 15, 2023 | ||||
| Edge |
2 | Pfui | 11:03 pm Jul 10, 2023 | ||||
| The X11 / Ubuntu Firefox/62.0 botnet is back and it's really persistent |
6 | SumGuy | 11:52 am Jul 8, 2023 | ||||
| Spike of Chrome robots.txt requests Unwitting scanning? |
7 | Pfui | 2:30 am Jul 3, 2023 | ||||
| OWLer |
2 | lucy24 | 7:38 pm Jun 27, 2023 | ||||
| htaccess avoidance Always read |
11 | Wayder | 7:10 am Jun 22, 2023 | ||||
| Googlebot smartphone crawl desktop URL 302 The 302 status code accounts for nearly half of the total |
9 | Jean23894 | 1:20 pm Jun 20, 2023 | ||||
| Thoughts on iCloud Private Relay? block, or not? |
1 | SumGuy | 10:58 pm Jun 13, 2023 | ||||
| Fuzz Faster U Fool v1.4.0-dev Hacker software |
5 | Bewenched | 12:52 am Jun 13, 2023 | ||||
| Was looking for info on a certain UA, found this. "Our API handles rotating proxies and headless browsers for you" |
5 | SumGuy | 12:51 am Jun 13, 2023 | ||||
| Clickagy Intelligence Bot v2 |
7 | Bewenched | 4:19 am May 27, 2023 | ||||
| GoogleOther crawler introduced by Google |
6 | phranque | 12:45 am May 27, 2023 | ||||
| Bots in Space |
9 | SumGuy | 4:01 pm May 22, 2023 | ||||
| Is Softbank an ISP or a Hosting company? |
3 | SumGuy | 6:15 pm May 19, 2023 | ||||
| Google-InspectionTool Crawler |
2 | engine | 1:20 pm May 18, 2023 | ||||
| Blocking everything except . Blocking everything except target country and good bots. |
5 | Wayder | 11:37 am May 17, 2023 | ||||
| I've stopped blocking 38.0.0.0/8 |
6 | SumGuy | 3:50 pm May 3, 2023 | ||||
| Persistent image scraper (jaunty) 179.60.147.xxx |
8 | Bewenched | 12:29 pm May 1, 2023 | ||||
| FemtosearchBot |
3 | Bewenched | 4:36 pm Apr 30, 2023 | ||||
| Spike of Blank User agents |
10 | Treud | 1:51 am Apr 28, 2023 |
