Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Microsoft IP (52.167.144.**) search.msn.com? |
2 | SumGuy | 4:18 pm Mar 31, 2023 | ||||
| VsuSearchSpider/1.0 seen today |
4 | SumGuy | 4:32 pm Mar 23, 2023 | ||||
| Screaming Frog SEO Spider/14.1 What is the use-case here? |
13 | SumGuy | 11:02 pm Mar 22, 2023 | ||||
| Curl, the URL fetcher that can, marks 25 years of transfers |
3 | tangor | 5:21 pm Mar 22, 2023 | ||||
| Apple Java bot |
7 | dstiles | 1:12 am Mar 21, 2023 | ||||
| sellers.json file seeing requests for it - what's it all about? |
7 | SumGuy | 4:41 pm Mar 19, 2023 | ||||
| Akamai and Linode relationship Akamai Connected Cloud? |
4 | SumGuy | 10:51 pm Mar 13, 2023 | ||||
| Accurate on-line service for VPN IP-identification? Is there one? |
1 | SumGuy | 2:27 pm Mar 2, 2023 | ||||
| Unknown UA verifies for a new social media site spoutible |
5 | dstiles | 12:18 am Feb 23, 2023 | ||||
| Multiple apple-touch-icon and favicon requests Icon requests |
11 | BeachWalker | 5:43 pm Feb 20, 2023 | ||||
| census.ipinfo.io Linode using Akamai as cover? |
3 | SumGuy | 10:57 pm Feb 16, 2023 | ||||
| Babbar's Barkrowler bot |
6 | blend27 | 3:32 pm Feb 15, 2023 | ||||
| Nmap Scripting Engine GET /nmaplowercheck |
2 | SumGuy | 3:15 pm Feb 12, 2023 | ||||
| Distributed Attack Mac Chrome/103.0.5060.134 |
10 | Pfui | 6:11 pm Feb 5, 2023 | ||||
| I let the domain name Expire accidentally, my bad :( Only to see who is really up to no good for the past 18 month. |
5 | blend27 | 10:48 pm Feb 4, 2023 | ||||
| Are we there Yet? RE: At Home with the Robots, 2022 Edition |
7 | blend27 | 9:00 pm Feb 4, 2023 | ||||
| Headers Trivia & Fun Who could tell me the problems with these headers sent to my site? |
7 | blend27 | 8:55 pm Jan 31, 2023 | ||||
| One of Oldies - my Favorites W3C//DTD HTML 4.01 Transitional//EN |
16 | blend27 | 6:13 pm Jan 21, 2023 | ||||
| Got a strange web-hit today GladysAction/1108 CFNetwork/1402.0.8 Darwin/22.2.0 |
6 | SumGuy | 3:11 pm Jan 16, 2023 | ||||
| leak.info/bot Mozilla/4.0 (compatible; fluid/0.0; +http://www.leak.info/bot.html) |
4 | SumGuy | 7:08 pm Jan 13, 2023 | ||||
| Does cloudflare hosts bots / scrapers? Is it normal to get web-hits from cloudflare IP's? |
4 | SumGuy | 12:05 pm Jan 13, 2023 | ||||
| "Moblie" ? New "Mozlila" typo in UA from MSFT in Redmond |
6 | Martin_Potter | 4:18 am Jan 13, 2023 | ||||
| Server Farms 2022 Continuing discussion of hosting and data center IP ranges |
14 | not2easy | 1:55 pm Jan 9, 2023 | ||||
| Blocking hits from TOR nodes Can go weeks without seeing them, then I see a lot on the same day |
11 | SumGuy | 3:57 am Jan 9, 2023 | ||||
| List of php script files requested by MSFT IP don't put these files on your website |
6 | SumGuy | 11:13 am Jan 8, 2023 |
