Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| One of Oldies - my Favorites W3C//DTD HTML 4.01 Transitional//EN |
16 | blend27 | 6:13 pm Jan 21, 2023 | ||||
| Got a strange web-hit today GladysAction/1108 CFNetwork/1402.0.8 Darwin/22.2.0 |
6 | SumGuy | 3:11 pm Jan 16, 2023 | ||||
| leak.info/bot Mozilla/4.0 (compatible; fluid/0.0; +http://www.leak.info/bot.html) |
4 | SumGuy | 7:08 pm Jan 13, 2023 | ||||
| Does cloudflare hosts bots / scrapers? Is it normal to get web-hits from cloudflare IP's? |
4 | SumGuy | 12:05 pm Jan 13, 2023 | ||||
| "Moblie" ? New "Mozlila" typo in UA from MSFT in Redmond |
6 | Martin_Potter | 4:18 am Jan 13, 2023 | ||||
| Server Farms 2022 Continuing discussion of hosting and data center IP ranges |
14 | not2easy | 1:55 pm Jan 9, 2023 | ||||
| Blocking hits from TOR nodes Can go weeks without seeing them, then I see a lot on the same day |
11 | SumGuy | 3:57 am Jan 9, 2023 | ||||
| List of php script files requested by MSFT IP don't put these files on your website |
6 | SumGuy | 11:13 am Jan 8, 2023 | ||||
| Is PsiNet a Host? Is it Okay to Block ALL their IP addys? |
5 | martinibuster | 8:26 pm Dec 26, 2022 | ||||
| Requesting opinion or experience with Centurylink /24 Bot activity? VPN or Proxy? |
1 | SumGuy | 11:19 pm Nov 28, 2022 | ||||
| Now I know what is behind these Kaspersky hits The Cerebro bot |
5 | SumGuy | 3:57 pm Nov 19, 2022 | ||||
| Something must be happening at Twitter First hit from the twitterbot in ages |
8 | SumGuy | 8:12 am Nov 19, 2022 | ||||
| AdobeUxTechC4-Async |
2 | Pfui | 4:59 pm Nov 18, 2022 | ||||
| moe New Google spawn? |
8 | Pfui | 12:36 am Nov 18, 2022 | ||||
| Any Downside to Blocking Amazonbot? |
14 | RubicCubed | 2:25 am Nov 16, 2022 | ||||
| CommonCrawl User-agent is just - CommonCrawl |
3 | SumGuy | 5:31 pm Oct 30, 2022 | ||||
| User agent : KOCMOHABT User agent of new browser under development |
14 | Martin_Potter | 9:32 pm Oct 29, 2022 | ||||
| Single Sign On + Identifying Search Bots question about websites switching to a metered paywall system |
13 | tube | 8:18 am Oct 16, 2022 | ||||
| Google and Bing Crawlers, User Agents |
2 | engine | 12:30 am Sep 30, 2022 | ||||
| Hetzner ASN / your-server.de bots |
3 | Ralph_Slate | 11:25 am Sep 28, 2022 | ||||
| Any Reason Not to Ban Oracle IP Range? DF Bot |
3 | martinibuster | 6:14 am Sep 26, 2022 | ||||
| MicrosoftPreview A new-ish bot |
1 | dstiles | 8:36 am Sep 13, 2022 | ||||
| What 3'rd party trust / security / AV-AM do you accept or block? And are you sure you know why they're hitting you? |
2 | SumGuy | 8:15 am Sep 13, 2022 | ||||
| User-agent: Mozlila (sic) Typo or attempt to look legit? |
5 | Martin_Potter | 11:29 pm Sep 12, 2022 | ||||
| robot clusters |
19 | lucy24 | 9:46 am Sep 4, 2022 |
