Forum Moderators: phranque
My site is currently being "nibbled on" by agodar and it is highly unnerving to see the hit numbers at 296 for March, down from 20,573 in February. What is going on here? We have been averaging 19,000 hits a month for the last year...
I have approached my hosting service on this and they are not willing to admit to making any changes to the site.
The site has a static IP.
The site on a shared Apache server.
We use our own scripts.
No blogs.
The hosting webmaster agrees that the site could have been hijacked but in a very loose sense of the word. A web spider (agodar) is apparently collecting information from pages without regard to the robots.txt file.
Our choices, according to the same webmaster, are to either put an .htaccess file in that explicitly denies access from our IP (we already have a 0 bytes one), or a firewall rule. But aside from the web spider not playing by the rules of fair play, they feel that nothing untoward is going on.
The owner/founder of the hosting service says that, if no unauthorized file changes were made (how can I tell?) and my only concern is that an Italian person or company is frequently checking the site, then this does not constitute a hijacking or hacking. He might be right, but I am feeling very vulnerable right now, with the hits being a small portion of what they have been for the last 12 months. I think he is covering his rear, frankly. (Am I allowed to submit a jpeg of the usage pie chart?)
I hope someone can give me some clear instructions as to how to proceed from here. I am a novice at this sort of thing but am willing to try anything you suggest to regain a sense of normality in my (web-site) life!
Thanks for reading my post.
this WebmasterWorld front page thread may have some information of interest:
Can you Trust your Host with your SEO? [webmasterworld.com]
First, the traffic has plummeted, and second, there's a rogue 'bot spidering the site.
These two issues are not related unless that rogue 'bot has copied your site and the copy now out-ranks your own on relevant searches (easy to check), though both are certainly serious.
For the traffic problem have a read through this thread [webmasterworld.com] -- an oldie, but still very much a goodie.
For controlling rogue 'bots on Apache, there are four basic approaches:
Here [webmasterworld.com] is one of many threads [google.com] here dealing with .htaccess code for blocking a specific user-agent. You or your programmer should be able to adapt it easily to your problematic visitor's user-agent name, as found in your raw server access log. Thorough testing will be needed to be sure that the block is effective, and that the proper server response code (403-Forbidden) is issued.
Having done so, you can then concentrate on optimizing your site and seeking relevant high-quality inbound links for search engine ranking, and promoting your site to market your products to a more targeted audience.
Jim
[edited by: jdMorgan at 4:57 am (utc) on Mar. 18, 2008]
Thank you for giving me such sterling advice (and in such clear language!).
I will contact our programmer this morning and ask him to institute the measures you have advocated.
I will certainly re-post once I have sorted this mess out and let you know how we did.
Once again, many thanks for your help.
Horsefern
Another thing to remember is bots do in DB sites cause the hits to climb at least in our sites I can tell when we have some heavy bot activity we will jump from 3k a week to 15 k a week in hits but the script counters remain constant or normal changes
Suggest you add a script counter (Statcounter or google) as well to see the sites more detailed performance.
Now as said above it is very well possible your traffic has been proxy hyjacked so using jdmorgan's advice is well worth looking into.
Do you have an alternate I can use that is as reliable? I have no reason to doubt the stats from Webalizer but a fresh program might show up some anomalies.
Thanks for the tip re Statcounter. I will investigate it forthwith.
Horsefern
