Welcome to WebmasterWorld Guest from 54.144.246.252

Forum Moderators: mademetop

How do I keep a link from being crawled?

   
4:13 pm on Apr 27, 2010 (gmt 0)

5+ Year Member



My business has to link a business we compete with for SERP's on a site we control...how do I do this so google/yahoo/bing won't count this link?

I put a rel='nofollow' on the linked image but they insisted we take this off. It's in a window.open function which they haven't complained about, but I hear google can crawl those links now, so I'm worried... Any other tricks?
9:08 pm on Apr 29, 2010 (gmt 0)

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



why would you let a competitor control how you implement links on your site?

send the link through a redirect script that is excluded by robots.txt.
9:15 pm on Apr 29, 2010 (gmt 0)

5+ Year Member



It's complicated...let's just say we share customers.

They would spot a redirect URL, so I don't think that would fly (they spotted the nofollow tag).

If I put the nofollow tag onto the link sneakily using something like ajax, would google still crawl the link?
6:03 am on May 3, 2010 (gmt 0)

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



i would go the other direction and ADD the link "sneakily" using javascript.
7:55 am on May 3, 2010 (gmt 0)

WebmasterWorld Senior Member topr8 is a WebmasterWorld Top Contributor of All Time 10+ Year Member



>>i would go the other direction and ADD the link "sneakily" using javascript.

google certainly follows links that are written with simple javascript, so you would probably need to call an external js file, or do something more complex than a straightforward document.write

there is only one sure way to do this, cloak the link, test for googlebot and don't serve the link to it, serve the link to everybody else.

OT i have links to lots of my competitors from my site and it doesn't seem to hurt me (or help them so much that they outrank me)
1:20 pm on May 3, 2010 (gmt 0)

5+ Year Member



If I cloaked the link (dynamically generated it if I didn't detect a search engine bot) couldn't that hurt my site in general? Can't search engines detect cloaking by pretending to be a non-search engine and comparing the results?

I like the javascript idea, but there is a good chance 'they' could find this (they found the nofollow attribute). Something like document.write('htt' + p://badsi' + 'te.com') should be enough to fake out google, right?
6:07 pm on May 3, 2010 (gmt 0)

WebmasterWorld Senior Member topr8 is a WebmasterWorld Top Contributor of All Time 10+ Year Member



cloaking is the only way, as it is the only way this other person won't see what you've done ... if they don't like nofollow they are not going to like js either

>>Something like document.write('htt' + p://badsi' + 'te.com') should be enough to fake out google, right?

it's easy enough to test this, but i'd say google could read and follow that.
6:18 pm on May 3, 2010 (gmt 0)

5+ Year Member



Wow...didn't know google was that good with javascript...think if that document.write was buried in say 5 nested functions, googlebot would get fed up and not crawl the js link?

Or if I rewrite the URL with javascript function from the header (like say to an internal redirect to mess with the link or to add a nofollow attribute dynamically) would google read this or the underlying html link?
7:13 pm on May 3, 2010 (gmt 0)

WebmasterWorld Senior Member topr8 is a WebmasterWorld Top Contributor of All Time 10+ Year Member



i don't know if that would work, i just know what i've tested for ... i have js linked bot traps which googlebot falls into all the time.

but my original point stands, if the guy is complaining about a nofollow link he surely is also going to complain about some kind of obscured js link too.
7:26 pm on May 3, 2010 (gmt 0)

5+ Year Member



Still paranoid about cloaking the link with PHP (which would be easy to do based on the user agent)...sure google wouldn't punish for that?
7:44 pm on May 3, 2010 (gmt 0)

WebmasterWorld Administrator buckworks is a WebmasterWorld Top Contributor of All Time 10+ Year Member



You could come at this from a completely different direction ... don't think about cutting off the flow of Page Rank to the competitor, think more in terms of reducing it.

Leave the link as a normal, dofollow HREF link, exactly what the competitor wants. But also add some other new links to the page, even if you just do it by adding few links to your internal navigation. That way the available link juice would be divided into more portions, and less would flow to the competitor's link.

Do the same thing a step further back in the chain. Place more links on pages that link to the page that houses the competitor's link. That again would re-channel the flow of link juice a bit.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month