Msg#: 4278128 posted 2:45 am on Mar 8, 2011 (gmt 0)
Hi folks, I am having an issue where many visitors to the site (not the majority) are requesting the RSS feed every time a page is loaded. Needless to say, that doubles the traffic and page view count of that user, causing security software to go ballistic.
In the head of most pages on the site, I have a tag like so:
Here is an example of a remote user-agent with the issue (not a bot):
Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; GTB6.6; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; InfoPath.1)
Looks like IE 8.
Perhaps there is some third-party add-on, toolbar, or reader software that might cause this behavior? Is there any way to fix it programmatically? I would rather leave the alternate link tags in place.... I suppose I could remove it from all subpages and leave it only on the homepage if that is better practice.
Msg#: 4278128 posted 4:54 pm on Jun 7, 2011 (gmt 0)
Thanks for the welcome, Bill. No, I did not ever find out what was causing this issue. It appears to have become less frequent.
Perhaps whatever bot/virus/malware/toolbar/reader that was causing the behavior on the users' end is less prevalent now.
My concern was that there may have been a problem with all of the custom code on my site (I wrote it all myself, and had a moment of self-doubt). However, the logs don't seem to show the issue recently. Just late 2010 through early 2011, from people all over the world that seem to be humans, not bots.
I saw maybe a dozen examples total during that time frame.
Logs looked like this:
[initial page load from google hit] [rss feed pulled] [next page view] [rss pulled] [next page view] [rss pulled] [next page view] [rss pulled] [rss pulled] [next page view] and so on.
I don't like banning people (bots are fine), so it seemed to be worth checking into, but will just have to remain a mystery.
Msg#: 4278128 posted 5:31 am on Jun 8, 2011 (gmt 0)
Thanks for the follow up!
Your hand written code certainly wasn't the issue, at least as far as that link in your <head> was concerned. You may have been on to something with the 3rd party problem, but I haven't seen it myself. Perhaps some of our members will run into this again and report in.