Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Google hiccup
3 wilderness 9:33 am Oct 14, 2019
  Hi
Pages exploit UA
15 Pfui 5:26 am Oct 5, 2019
  URI: ./ ./ mnt/ custom/ ProductDefinition
DVR remote code execution
3 Pfui 5:35 pm Sept 27, 2019
  Yisou again
2 lucy24 12:06 am Sept 27, 2019
  GrumpyCrawler
Microsoft range
3 Pfui 6:02 pm Sept 24, 2019
  Mb2345Browser/9.0
I'm pretty sure this is actually a crawler, not a browser
8 notriddle 2:51 am Sept 10, 2019
  serpstatbot/1.0 (advanced backlink tracking bot
3 notriddle 9:28 pm Sept 3, 2019
  Moreover
4 lucy24 1:18 am Sept 2, 2019
  BingPreview from Facebook
4 lucy24 8:50 pm Sept 1, 2019
  GoogleImageProxy
redux; runs amok
9 Pfui 11:22 am Aug 30, 2019
  looid.com Search Engine
3 Pfui 4:54 am Aug 29, 2019
  Plague of Viewbots
help on how to detect view bots?
12 PeachyCheech 8:47 pm Aug 22, 2019
  Looking at some strange hits from Microsoft IP's
Some are from outlook outbound mail servers?
2 SumGuy 6:27 am Aug 21, 2019
  Firefox on Mac User-agent -> version number is 4-digit hex value
Seeing coordinated bot activity
4 SumGuy 1:38 am Aug 16, 2019
  User Agent cdnposUserAgentcdnpos<>
3.224.211.xxx
2 Bewenched 8:22 pm Aug 13, 2019
  Seeing hits from 3.0.0.0/8 for first time
Is Amazon
9 SumGuy 7:15 pm Aug 13, 2019
  Bad Bot hack attempts 23.83.128.0/21
5 Bewenched 5:45 pm Aug 13, 2019
  Instapaper/7.7.1.2 CFNetwork/978.0.7 Darwin/18.6.0
3 tangor 3:37 am Aug 3, 2019
  newspaper
9 Pfui 10:32 pm July 29, 2019
  Facebook pretending to be Bing
Bing UA from Facebook IP
8 dstiles 10:21 am July 29, 2019
  ?source=post page---------------------------
What the--?
6 Pfui 6:24 am July 29, 2019
  Chrome Lighthouse redux
3 lucy24 12:13 am July 29, 2019
  Safe to Block HeadlessChrome?
9 glakes 1:16 am July 26, 2019
  two from Facebook
13 lucy24 5:39 pm July 25, 2019
  Another funny fake user agent string
5 notriddle 5:48 pm July 21, 2019