homepage Welcome to WebmasterWorld Guest from 54.227.146.68
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Marketing and Biz Dev / General Search Engine Marketing Issues
Forum Library, Charter, Moderators: mademetop

General Search Engine Marketing Issues Forum

    
How do I keep a link from being crawled?
smithaa02




msg:4122957
 4:13 pm on Apr 27, 2010 (gmt 0)

My business has to link a business we compete with for SERP's on a site we control...how do I do this so google/yahoo/bing won't count this link?

I put a rel='nofollow' on the linked image but they insisted we take this off. It's in a window.open function which they haven't complained about, but I hear google can crawl those links now, so I'm worried... Any other tricks?

 

phranque




msg:4124643
 9:08 pm on Apr 29, 2010 (gmt 0)

why would you let a competitor control how you implement links on your site?

send the link through a redirect script that is excluded by robots.txt.

smithaa02




msg:4124653
 9:15 pm on Apr 29, 2010 (gmt 0)

It's complicated...let's just say we share customers.

They would spot a redirect URL, so I don't think that would fly (they spotted the nofollow tag).

If I put the nofollow tag onto the link sneakily using something like ajax, would google still crawl the link?

phranque




msg:4125949
 6:03 am on May 3, 2010 (gmt 0)

i would go the other direction and ADD the link "sneakily" using javascript.

topr8




msg:4125967
 7:55 am on May 3, 2010 (gmt 0)

>>i would go the other direction and ADD the link "sneakily" using javascript.

google certainly follows links that are written with simple javascript, so you would probably need to call an external js file, or do something more complex than a straightforward document.write

there is only one sure way to do this, cloak the link, test for googlebot and don't serve the link to it, serve the link to everybody else.

OT i have links to lots of my competitors from my site and it doesn't seem to hurt me (or help them so much that they outrank me)

smithaa02




msg:4126068
 1:20 pm on May 3, 2010 (gmt 0)

If I cloaked the link (dynamically generated it if I didn't detect a search engine bot) couldn't that hurt my site in general? Can't search engines detect cloaking by pretending to be a non-search engine and comparing the results?

I like the javascript idea, but there is a good chance 'they' could find this (they found the nofollow attribute). Something like document.write('htt' + p://badsi' + 'te.com') should be enough to fake out google, right?

topr8




msg:4126255
 6:07 pm on May 3, 2010 (gmt 0)

cloaking is the only way, as it is the only way this other person won't see what you've done ... if they don't like nofollow they are not going to like js either

>>Something like document.write('htt' + p://badsi' + 'te.com') should be enough to fake out google, right?

it's easy enough to test this, but i'd say google could read and follow that.

smithaa02




msg:4126273
 6:18 pm on May 3, 2010 (gmt 0)

Wow...didn't know google was that good with javascript...think if that document.write was buried in say 5 nested functions, googlebot would get fed up and not crawl the js link?

Or if I rewrite the URL with javascript function from the header (like say to an internal redirect to mess with the link or to add a nofollow attribute dynamically) would google read this or the underlying html link?

topr8




msg:4126309
 7:13 pm on May 3, 2010 (gmt 0)

i don't know if that would work, i just know what i've tested for ... i have js linked bot traps which googlebot falls into all the time.

but my original point stands, if the guy is complaining about a nofollow link he surely is also going to complain about some kind of obscured js link too.

smithaa02




msg:4126319
 7:26 pm on May 3, 2010 (gmt 0)

Still paranoid about cloaking the link with PHP (which would be easy to do based on the user agent)...sure google wouldn't punish for that?

buckworks




msg:4126330
 7:44 pm on May 3, 2010 (gmt 0)

You could come at this from a completely different direction ... don't think about cutting off the flow of Page Rank to the competitor, think more in terms of reducing it.

Leave the link as a normal, dofollow HREF link, exactly what the competitor wants. But also add some other new links to the page, even if you just do it by adding few links to your internal navigation. That way the available link juice would be divided into more portions, and less would flow to the competitor's link.

Do the same thing a step further back in the chain. Place more links on pages that link to the page that houses the competitor's link. That again would re-channel the flow of link juice a bit.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Marketing and Biz Dev / General Search Engine Marketing Issues
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved