noindex, follow - potential risks?

[jaymax182]

Hey Guys,

I was wondering if you all know of any risks with the "no index, follow" in terms of other search bots ignoring the instruction.

as far as i understand, google, yahoo and msn wont index the pages - but what about those other bots that could potentially ignore your request.

Is there a way to redirect people but allow the search engines to crawl the page...or is that considered cloaking?

the pages i want to do a "no index, follow" are very basic non-important parts of the site that has some links on them.

Your help would be appreciated.


Jay

[goodroi]

as far as seo goes you really only need to worry about google and microsoft. they power almost all searches in the us.

yahoo is now powered by microsoft. aol is powered by google. ask network just announced they are stopping their crawler.

[tedster]

Welcome to the forums, jaymax182.

Note that a robots meta tag of any kind can only be read and obeyed if the page is crawled in the first place - so you definitely want the crawlers to request the page. If you redirect other users but not the search engine crawlers, then that is cloaking.

Anytime spiders and human users get different treatment that is cloaking, by definition. Whether it is considered DECEPTIVE cloaking and therefore penalized is a slightly different question.

I'm not clear why these pages even exist, if you don't want users to see them - could you explain it a bit more more?

[jaymax182]

thanks tedster, i'm glad to be on here. Very informative stuff that i've been reading on some of these threads.

thank you both for the response as it pretty much clears this up for me, especially regarding the rules of the search engines.

the reason i wanted to keep those pages from the user is because there's no content on them except for a few links to a couple other sites that i have so i felt it wouldn't be necessary to have it displayed on the main website. so i guess what i'll go ahead and do is simply a 'no index, follow' on the page itself. a lot of this stuff is new to me so i just wanted to make sure that i was 100% compliant.

thanks again for the responses guys.

[tedster]

It still doesn't really gel for me. Here's what I get:

1. You don't want visitors on those pages.
2. You don't want search engines to index those pages.

So the big question for me - why are they even on the site at all? It sounds like they just don't belong.

[jaymax182]

they're simply pages that i've placed links on.

I dont want the search engines to index them but i want the links to count which is why i was thinking 'no index, follow'.

i had read somewhere that some search engines out there ignores the 'no index, follow' instruction so that's why i had initially asked about it....but goodroi pretty much made it clear that those other engines wouldn't really matter since everyone uses google and microsoft either way.

[tedster]

If the pages shouldn't be indexed and website visitors shouldn't see them, then that sounds like something done only to manipulate rankings. No matter how you handle it technically, it sounds deceptive to me and the kind of thing Google wouldn't like.

I'd say find a completely up front way to present those links or don't offer them at all.

[jaymax182]

ok, i see what you're saying. so would you say it's best to simply be dependent on submitting to directories in order to build your rankings?

[Robert Charlton]

jaymax182 - I gather from the above that these are links intended to help your own rankings.

Are these...

a) links to other sites you own?
or...
b) are they links to independent sites as part of a link exchange program?

[jaymax182]

yes, they're actually links to other sites that i own. they're about 5 sites that i own and was looking to link to them - that's why i didn't want the user to see them because the contents are fairly dissimilar.

[AussieWebmaster]

1. the dissimilarity won't help with lift much
2. how are you passing link juice to these pages? hidden in sitemap? if no pagerank no juice
3. sooner or later this will get seen and cause problems - if you start doing well in your space competitors will do research on your sites and find this and could send spam report to google

[jaymax182]

i've heard something to that effect...that's why i wanted to do a 'no index, follow'. That's also the reason i had asked about those bots that ignore the no index, follow instructions. which would enable your competitors to see exactly those pages that i was trying to not display.

that's why i was curious to know whether i could do the no index, follow and do a redirect if the situation does come up where those pages end up where people can actually see them - but that's cloaking apparantly.

[tedster]

Let me give you some straight talk. This is not a good way to link to your other sites - just don't do it. Either link in the open when it's useful for your visitors, or don't link at all. All these technically manipulative approaches are very likely to get you into trouble eventually.

The risk is everything - all your work. So be direct about your linking and your marketing.

[anallawalla]

I'm not clear why these pages even exist, if you don't want users to see them - could you explain it a bit more more?

For a large directory, this technique can be used for page 2 onwards of the paginated index, where you want the crawler to follow the links to the lower levels, but not index the rest of the paginated index. We found that the index pages used to be indexed first and began to rank for anchor text on those pages, rather than the pages those links pointed to.

[jaymax182]

and the reason for not doing it at all is because the content are dissimilar? so for example, what if i get other people who own websites with similar content - they link to me - i like to them. so therefore the links that i have would be linking to other people's sites with very similar content as opposed to content that doesn't relate. Can you still get in trouble for that - or does google hate when you have links to other sites without merging it into the actual page content?

[tedster]

You can link as you like, but in-content links have more punch now, under Google's "reasonable surfer" model.

[harsh2k218]

google won't count this as clocking, you don't worry about that.

[jaymax182]

awesome - thanks for all the feedback guys.

[jaymax182]

one day i hope to be as knowledgeable as you tedester :)