homepage Welcome to WebmasterWorld Guest from 54.237.54.83
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

This 31 message thread spans 2 pages: < < 31 ( 1 [2]     
HTTrack / Virgin Media
cyberdyne




msg:4420614
 7:00 pm on Feb 22, 2012 (gmt 0)

My site received a visit from a Virgin Media customer during which five hits from the U-A: HTTrack appeared hitting my robots, site root and one thumbnail image. It initially appeared as though the user was attempting to download my site or maybe Virgin Media themselves were trying to cache my site but it seems unusual for such a large ISP to use such a tool.

As the user registered on my site gallery I emailed them to ask if they knew anything about the activity and they genuinely seemed clueless as to why HTTrack was used, or even what it was.

Has anyone seen this sort of activity before, specifically from Virgin Media, and is it un/common for an ISP to use a commercial tool such as this?

Thanks in advance

77.100.245.xx - - [22/Feb/2012:03:50:44 +0000] "GET /robots.txt HTTP/1.1" 200 6265 "-" "Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)"
cpc18-nmal16-2-0-custxx.19-2.cable.virginmedia.com - - [22/Feb/2012:03:50:44 +0000] "GET / HTTP/1.1" 403 1343 "-" "Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)"
77.100.245.xx - - [22/Feb/2012:03:51:48 +0000] "GET /robots.txt HTTP/1.1" 200 6265 "-" "Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)"
cpc18-nmal16-2-0-custxx.19-2.cable.virginmedia.com - - [22/Feb/2012:03:51:49 +0000] "GET / HTTP/1.1" 403 1343 "-" "Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)"
cpc18-nmal16-2-0-custxx.19-2.cable.virginmedia.com - - [22/Feb/2012:03:52:26 +0000] "GET /gallery/thumb_001.jpg HTTP/1.1" 403 1412 "-" "Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)"

 

cyberdyne




msg:4422193
 10:10 am on Feb 27, 2012 (gmt 0)

*** connects only to those servers upon which target Web sites operate. It will, however, also connect to a secure server provided by *** at timely pre-set intervals, to validate and verify the IP addresses of the computers running the *** software. This information is used to prevent target Web sites from “blocking” the end user’s true IP address, preventing them from accessing and gathering the needed data. The program can also switch to a “secured” model to hide the real IP address of the executing computer if necessary. Furthermore, the *** application will also continuously verify the data extraction process, to ensure the smoothest, problem-free operations. Invalid or unsuccessful searches can be automatically reactivated, for example, by using an alternative routing proxy server to re-access the target Web site. When this function is executed, no Web sites other than the one targeted for data retrieval will be accessed. This function can be manually disabled by users, for various computer and network security purposes.


Persistent little buggers aren't they.

This 31 message thread spans 2 pages: < < 31 ( 1 [2]
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved