Forum Moderators: open
218.****.xx.250 - - [22/Jul/2004:18:39:51 -0400] "GET / HTTP/1.1" 301 263 "-" "Mozilla/4.0 (compatible; Cerberian Drtrs Version-3.1-Build-16)"
Jim
[edited by: volatilegx at 4:50 pm (utc) on July 23, 2004]
[edit reason] obscured IP address [/edit]
On June 30, I had this UA from two entirely differnt US IP's ( Apsecure Technologies & RAGING WIRE FON ) grabbing the same pages consecutively from both IP's on two of my sites.
If you desire the IP's, sticky me.
Don
After doing some research I have found that their technology is used by quite a few firewalls such as SonicWALL, proxy products like MS ISA Server, and software firewalls like ZoneAlarm.
They have service points in Sacramento, Salt Lake City, Japan and UK which may be why you are seeing them come from Taiwan (?)
I also noticed that they are not actually crawling. They ask for things like www.mycoolsite.com/img which must be preconfigured in the software because it doesn't exist on my server. Their visiting is often triggered by a visitor to my site. Apparently the software can make decisions on the fly.
If they are unable to categorize the site it will be automatically denied to the surfer by the employer-no matter what. At lease if the site is categorized and if the employer says the category can be surfed during work hours I have half a chance.
My 2 cents.
There was probably a 301 redirect involved though I didn't check the actual log files, but I do have that running.
I was doing some testing of a mobile application we are developing and so was watching the action.
I was using Vodaphone in the UK. It made a request right before the phone connected. The phone request definately triggered it because it was for a test script i was using.
GATEWAY_INTERFACE : CGI/1.1
HTTP_ACCEPT : text/plain, text/html, text/xml, text/vnd.wap.wml, application/vnd.wap.wmlc
HTTP_ACCEPT_LANGUAGE : en, *;q=0.7
HTTP_CONNECTION : Keep-Alive
HTTP_USER_AGENT : Mozilla/4.0 (compatible; Cerberian Drtrs Version-3.1-Build-16)
Does that help anybody?
Vodafone has introduced content control in the UK a few weeks ago, not sure about other countries and/or operators.
HTH,
Mario.
Especially since the last submission are time consecutive attempts from two different continents.
Although I personally have the UA denied access, I'm more inclined to believe Ciberian to be a harvester rather than a useful tool.
All I can say is that I have replicated this with my own mobile - access my site with content filtering "on" will show these entries, switching it off will make them disappear.
[cerberian.com...] may be useful if someone feels like they can be bothered to follow up on it and find out what exactly they do.
There's no pattern.
The last submission to me hardly allows for any use related to mobiles phones as previously suggested.
I'd have to agree with you, Don. Why post about it though? The last entry doesn't look suspicious to me in any way, other than the fact that it requests the same file as the preceeding Cerberian entry. Note the preceeding entry is 12 seconds earlier.
Successive log enries/requests for this page (and many others like it) on one of my sites are NOT conicidence. Especially when the 1st request was denied from RIPE and the suceeding request was allowed from ARIN.
Had my logs reflected a multitude of requests either that same day or over a few days (this happens frequently in many of the page/articles I have online,) than it would provide some possible connection to a news release or similar mention which might prompt internet searches for this specific content, that was not the case in this instance.
As a result, the only conclusion for me to draw is that even though on the surface you and I might percieve these two visits to be unrealated, they in fact are related and the same visitor. With different IP's from different continents.
Don
