Welcome to WebmasterWorld Guest from 54.144.80.75

Message Too Old, No Replies

Google indexing /feed URLs

Google is indexing my wordpress site as www.domain.com/post/feed

     
9:27 pm on Sep 15, 2006 (gmt 0)

New User

10+ Year Member

joined:Sept 1, 2005
posts:14
votes: 0


Hello,
Iīve got a couple of wordpress blogs that Iīve been keeping for some time now (one is 1year+ and the other one about 8 months).Both sites have a lot of indexed pages,but for some reason google has been indexing them as www.domain.com/post/feed/ instead of just www.domain.com/post and they are all supplemental. Iīm talking about 500 indexed /feed pages.
Most of the traffic coming in from google is now coming in to these /feed urls and thatīs no good since that is just an xml feed and they just leave.
Both sites have updated sitemaps and I fixed the identical meta description on pages around 2 weeks ago (which I thought was causing the supplemental results problem).
Any ideas about what might be going on?
Thanks!
11:51 pm on Sept 15, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 5, 2002
posts:150
votes: 0


I don't know, but I've run into it quite a bit on the searching side. I'll find a result that looks interesting, and click to find myself within an unintelligible jumble of XML.

As Nero Wolfe would say, Pfui.

11:56 pm on Sept 15, 2006 (gmt 0)

Senior Member from KZ 

WebmasterWorld Senior Member lammert is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 10, 2005
posts:2886
votes: 1


Google is indexing every readable content it finds, including XML files. The nice thing is that Google recognizes wildcards in robots.txt. You can add the following rule to your robots.txt:

User-agent: Googlebot
Disallow: /*/feed/$

This solved the problem for my WordPress based blog. You may have to wait one year before the supplemental results are gone. Google is slow with cleaning the supplemental index.

12:47 am on Sept 16, 2006 (gmt 0)

New User

10+ Year Member

joined:Sept 1, 2005
posts:14
votes: 0


Thanks lammert Iīm gonna give that a shot.
I have been searching for other blogs being affected by this and I havenīt found any, so I donīt know if itīs some setting in particular I set in wordpress or what is making this happen
1:49 am on Sept 16, 2006 (gmt 0)

Junior Member

5+ Year Member

joined:Aug 17, 2006
posts:61
votes: 0


Yep, I stumbled onto this same problem about three months ago. It's not just particular to your site.

You might also check to see if Google is indexing your trackback URLs. I ultimately fixed it with a robots.txt file:

Disallow: /*/*/*/*/feed/
Disallow: /*/*/*/*/trackback/

Good luck!

3:51 am on Sept 16, 2006 (gmt 0)

New User

10+ Year Member

joined:Sept 1, 2005
posts:14
votes: 0


Thanks Elvis!
Anyone have any idea how long it might take google to remove these results from the index?
On one of the sites Iīm even seeing a bunch of those registrar parking pages! I regged the domain and I forgot to change the DNS while I got to using that domain and now there are like 20 results for www.example.com/?q=Hotel+Suites+Weddings and stuff like that.The worst part is that they actually go to the index page of the blog, is there actually any way to remove those results?
1:40 am on Sept 28, 2006 (gmt 0)

Junior Member

5+ Year Member

joined:Aug 30, 2006
posts:76
votes: 0


Easy answer for the future is simply to remove this feed link url from single.php. Then google will never find it.

I noticed the same thing, and this link is not important to me, so I simply deleted it.

1:45 am on Sept 28, 2006 (gmt 0)

Junior Member

5+ Year Member

joined:Aug 30, 2006
posts:76
votes: 0


"there are like 20 results for www.example.com/?q=Hotel+Suites+Weddings and stuff like that.The worst part is that they actually go to the index page of the blog, is there actually any way to remove those results? "

If someone knows the answer to this, let me know! I have almost the very same problem.

5:54 am on Oct 6, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 19, 2004
posts:1939
votes: 0


Goto the Apache thread and do some reading on the information JD Morgan and friends have posted. I have found the answer there.
6:02 am on Oct 6, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


Is this the thread?

[webmasterworld.com...]

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members