Welcome to WebmasterWorld Guest from

Forum Moderators: martinibuster

Message Too Old, No Replies

Yahoo index www and non www domain

How Can I Get Yahoo to Toss the Wrong One and Keep the Right One?



3:30 pm on Oct 17, 2005 (gmt 0)

10+ Year Member

hi all, for my sites yahoo had indexed both the www and non www versions of the name

www. domain. com
domain .com

A have a 301 redirect in htaccess that has worked fine for other spiders but not yahoo.

Is there another way to get rid on the non-www version in yahoo?



6:24 am on Oct 25, 2005 (gmt 0)

10+ Year Member

I'm not exactly sure how to do it (have not got the time right at this moment) but I think you probably could use .htaccess to serve different robots.txt.

You could redirect all requests for robots.txt that come in without the www to a cloaked version that basically disallows *

Something like

RewriteCond %{HTTP_HOST} ^example\.com
RewriteCond %{REQUEST_URI} robots.txt
RewriteRule ^(.*)$ <this is the bit I haven't worked out> [R=200]

I have no idea if it matters that you send it to (say) http://www.example.com/special-robots.txt. You may need to send it to something called 'robots.txt' in a sub domain?

Try it using a file named something other than robots.txt and test it with a "header checking" utility.

You could perhaps post a slightly reworded question in the Apache forum [webmasterworld.com]


5:27 am on Oct 26, 2005 (gmt 0)

10+ Year Member

I got curious so I asked the question myself in the Apache forum [webmasterworld.com].


5:49 am on Oct 26, 2005 (gmt 0)

10+ Year Member

How long ago was the 301 put into place? It can take a while (4-6 weeks in my experience) for this to fully take effect.


6:19 am on Oct 26, 2005 (gmt 0)

10+ Year Member

(4-6 weeks in my experience)

I agree - sort of - 4 to 6 weeks for it to find the new page but another 6 to 18 months to stop looking for the original!

(Perhaps I misunderstood the original post - now I re-read it. I assumed that yahoo had found the new pages and was now doing it's customary look and relook and relook etc and non-removal of the old.)


6:14 pm on Nov 1, 2005 (gmt 0)

10+ Year Member

ah, patience is needed i suppose :-)

thansk all


Featured Threads

Hot Threads This Week

Hot Threads This Month