Welcome to WebmasterWorld Guest from 54.167.46.29

Forum Moderators: bill & werty

Message Too Old, No Replies

Duff feed URL with unwanted amp; part within.

Googlebot redirected.

     
10:18 pm on Jun 27, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


The vast majority of subscribers have subscribed via this URL:

www.example.com/pagename?feed=atom&param=value



When this User Agent visits the site, it reports the number of subscribers and the feed ID as part of the UA string:

"Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; nnn subscribers; feed-id=nnnnnnnnnnnnnnnnnnn)"



However the same UA also requests this other malformed URL:

www.example.com/pagename?feed=atom&param=value


from time to time, and it has a very small number of subscribers. This URL has amp; in it, not just the & part.


A few weeks ago a 301 redirect was installed to remove the "amp;" part of the requested URL. Google appears to follow the redirect requesting both URLs in turn; the one with "amp;" followed immediately by the one without it, both using the feed ID with the small number of subscribers.


The point is, the "faulty" URL continues to be requested, and now there are two separate feed IDs that request the exact same URL, one natively and the other redirected.


How long does it take for the redirected URL to no longer be requested? Does it require the subscribers to alter something in their feed reader, perhaps unsubscribe and resubscribe, or will things eventually update themselves automatically?
6:41 am on Sept 27, 2011 (gmt 0)

Administrator from JP 

WebmasterWorld Administrator bill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Oct 12, 2000
posts: 14470
votes: 49


Are you still experiencing this?
7:14 am on Sept 27, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


Yes. I tried redirecting for several months. The requests for the wrong URL version continued several times per day.

For the last month or two I have been rewriting the request to a static XML file with one message in it "Please unsubscribe from this feed and resubscribe using the URL version without amp; in it". The requests continue and the subscriber number remains static.

I also tried something else for about one month in between those two, but can't remember what it was.
7:36 am on Sept 27, 2011 (gmt 0)

Administrator from JP 

WebmasterWorld Administrator bill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Oct 12, 2000
posts: 14470
votes: 49


Obviously the fetcher doesn't recognize the redirect then. I guess you've tried 404 or 410 errors as well?
8:32 am on Sept 27, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


That might be what I tried in the month that I can't remember what I did. The requests kept on coming whatever it was I did.

There's a good reason for not using 404 or 410. I don't want to lose the subscribers. I have assumed that if they see those types of error, they will assume the site has closed down.

Sending a "please resubscribe using the correct URL" message hasn't worked, so maybe I'll revert to using the 301 redirect instead.
1:45 am on Sept 28, 2011 (gmt 0)

Administrator from JP 

WebmasterWorld Administrator bill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Oct 12, 2000
posts: 14470
votes: 49


From [google.com...]
Feedfetcher does not follow robots.txt guidelines.

If your feed is publicly available, Google can't restrict users from accessing it. One solution is to configure your site to serve a 404, 410, or other error status message to user-agent Feedfetcher-Google.


It's understandable that you don't want to lose the subscribers. Maybe adding a large eye-catching image to the Please unsubscribe message might draw more attention.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members