homepage Welcome to WebmasterWorld Guest from 107.22.78.233
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Yahoo / Yahoo Search Engine and Directory
Forum Library, Charter, Moderators: martinibuster

Yahoo Search Engine and Directory Forum

    
Yahoo index www and non www domain
How Can I Get Yahoo to Toss the Wrong One and Keep the Right One?
frritchi




msg:832151
 3:30 pm on Oct 17, 2005 (gmt 0)

hi all, for my sites yahoo had indexed both the www and non www versions of the name

ie
www. domain. com
domain .com

A have a 301 redirect in htaccess that has worked fine for other spiders but not yahoo.

Is there another way to get rid on the non-www version in yahoo?

thanks
Frank

 

fish_eye




msg:832152
 6:24 am on Oct 25, 2005 (gmt 0)

I'm not exactly sure how to do it (have not got the time right at this moment) but I think you probably could use .htaccess to serve different robots.txt.

You could redirect all requests for robots.txt that come in without the www to a cloaked version that basically disallows *

Something like
RewriteCond %{HTTP_HOST} ^example\.com
RewriteCond %{REQUEST_URI} robots.txt
RewriteRule ^(.*)$ <this is the bit I haven't worked out> [R=200]

I have no idea if it matters that you send it to (say) http://www.example.com/special-robots.txt. You may need to send it to something called 'robots.txt' in a sub domain?

Try it using a file named something other than robots.txt and test it with a "header checking" utility.

You could perhaps post a slightly reworded question in the Apache forum [webmasterworld.com]

fish_eye




msg:832153
 5:27 am on Oct 26, 2005 (gmt 0)

I got curious so I asked the question myself in the Apache forum [webmasterworld.com].

marcs




msg:832154
 5:49 am on Oct 26, 2005 (gmt 0)

How long ago was the 301 put into place? It can take a while (4-6 weeks in my experience) for this to fully take effect.

fish_eye




msg:832155
 6:19 am on Oct 26, 2005 (gmt 0)

(4-6 weeks in my experience)

I agree - sort of - 4 to 6 weeks for it to find the new page but another 6 to 18 months to stop looking for the original!

(Perhaps I misunderstood the original post - now I re-read it. I assumed that yahoo had found the new pages and was now doing it's customary look and relook and relook etc and non-removal of the old.)

frritchi




msg:832156
 6:14 pm on Nov 1, 2005 (gmt 0)

ah, patience is needed i suppose :-)

thansk all

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Yahoo / Yahoo Search Engine and Directory
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved