homepage Welcome to WebmasterWorld Guest from 54.196.198.213
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Website
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Disallow Yahoo to block site explorer
minnapple




msg:3723378
 10:47 pm on Aug 14, 2008 (gmt 0)

Has anyone blocked yahoo slurp to hide backlinks from Yahoo! Site Explorer.

The reason I ask, I am wondering if Yahoo!
1. removes old backlinks from Site Explorer searches.
[ I know they are slow to remove old link data, but do they eventually remove it? ]
2. honors the disallow when backlinks are considered in general.

So far I have not seen a much of impact of disallowing link pages from Yahoo! in terms of rankings to target sites. However it has only been a month since I have disallowed Y!.

I would think I should know in about 5 months, but if anyone is willing to give up their personal past experiences, I will follow up with mine.

 

goodroi




msg:3723823
 2:08 pm on Aug 15, 2008 (gmt 0)

If you are looking for a site that has blocked Yahoo (and MSN) then check out schools.nyc.gov/robots.txt. They are not doing for link issues like you but for bandwidth issues (which imho is silly).

You do bring up a good point about disallowing links from Yahoo. Historically they have treated nofollow tags differently than Yahoo. I would expect Yahoo to respect a robots.txt disallow. It just might take some time for them to make the change (especially if you are disallowing a very deep page). Once they make the change will they remove the link data from their database or leave it their since slurp is blocked from confirming the links are removed? Looking forward to figuring this one out.

Lord Majestic




msg:3723827
 2:16 pm on Aug 15, 2008 (gmt 0)

This (if it works at all) will only hide internal backlinks from your site in Site Explorer - if you are afraid of the competition then it is the external links to your site that they would be most interested in anyway and those are not under your control (unless they come from your own additional sites).

In all probability (and I'd say certainty) backlinks index is a separate one that gets additions from recent crawls - this means that if you prevent your site from being crawled then the best you can achieve is that no new backlinks (from your sites) will be added.

Yahoo shows nofollow marked links in Site Explorer just fine (Google appears to do the same).

minnapple




msg:3730061
 10:08 pm on Aug 24, 2008 (gmt 0)

To this date, any new links placed on sites that have been disallowed, do not show up when using site explorer.

Old link data still exists.
I will repost if old data is flushed.

minnapple




msg:3783841
 2:12 am on Nov 11, 2008 (gmt 0)

four months to flush

Lord Majestic




msg:3783846
 2:31 am on Nov 11, 2008 (gmt 0)

Do you still see external backlinks shown for your site found on websites where you did not disallow YSE for whatever reason (like when the website owned by other people)?

minnapple




msg:3784276
 6:17 pm on Nov 11, 2008 (gmt 0)

Yes

Lord Majestic




msg:3784298
 6:56 pm on Nov 11, 2008 (gmt 0)

Ok, so in effect all you achieved is removed from YSE backlinks founds on sites that you personally control and it took almost 4 months to do so, where as external backlinks from other sites not under your control are still being shown in YSE.

minnapple




msg:3786034
 2:23 am on Nov 14, 2008 (gmt 0)

Lord Majestic
yes
like your bot :)
can you comp me an extended trial?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved