Yahoo index www and non www domain

How Can I Get Yahoo to Toss the Wrong One and Keep the Right One?

[frritchi]

hi all, for my sites yahoo had indexed both the www and non www versions of the name

ie
www. domain. com
domain .com

A have a 301 redirect in htaccess that has worked fine for other spiders but not yahoo.

Is there another way to get rid on the non-www version in yahoo?

thanks
Frank

[fish_eye]

I'm not exactly sure how to do it (have not got the time right at this moment) but I think you probably could use .htaccess to serve different robots.txt.

You could redirect all requests for robots.txt that come in without the www to a cloaked version that basically disallows *

Something like

RewriteCond %{HTTP_HOST} ^example\.com
RewriteCond %{REQUEST_URI} robots.txt
RewriteRule ^(.*)$ <this is the bit I haven't worked out> [R=200]

I have no idea if it matters that you send it to (say) http://www.example.com/special-robots.txt. You may need to send it to something called 'robots.txt' in a sub domain?

Try it using a file named something other than robots.txt and test it with a "header checking" utility.

You could perhaps post a slightly reworded question in the Apache forum [webmasterworld.com]

[fish_eye]

I got curious so I asked the question myself in the Apache forum [webmasterworld.com].

[marcs]

How long ago was the 301 put into place? It can take a while (4-6 weeks in my experience) for this to fully take effect.

[fish_eye]

(4-6 weeks in my experience)

I agree - sort of - 4 to 6 weeks for it to find the new page but another 6 to 18 months to stop looking for the original!

(Perhaps I misunderstood the original post - now I re-read it. I assumed that yahoo had found the new pages and was now doing it's customary look and relook and relook etc and non-removal of the old.)

[frritchi]

ah, patience is needed i suppose :-)

thansk all