Forum Moderators: open

Message Too Old, No Replies

One specific page I don't want indexed...

         

lazyz

5:14 pm on Sep 25, 2002 (gmt 0)

10+ Year Member



I have a page on my site full of links that really needs to be accessed by my customers but not by Google's spider... I don't want my site to be put at risk as becoming a link farm but at the same time I want to provide the list of links to my customers. What is the best way to approach this? .htaccess(good code anyone?) or a line in the header(a good line anyone)? I don't want to risk Googlebot not indexing the entire site either...

gmoney

5:21 pm on Sep 25, 2002 (gmt 0)

10+ Year Member



You may want to check out:
[searchengineworld.com...]

I think what you are looking for is the following but please don’t just take my word for it:)
User-agent: googlebot
Disallow: filename.htm

added: There are probably other, perhaps better, ways to do it but this is the way I know about.

rogerd

5:25 pm on Sep 25, 2002 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



You could add a ROBOTS NOINDEX meta tag to back up the robots.txt prohibition, too.

If any of these are reciprocal links, of course, your linkees may not be happy if they are astute enough to check your page and/or robots.txt file. It's kind of an unwritten assumption that if you are exchanging links you are exchanging spiderable, indexable links.

bobmark

5:35 pm on Sep 25, 2002 (gmt 0)

10+ Year Member



I like the "noindex,nofollow" metatag and Googlebot seems to have no probs with it. On my site I have pages that have to be "de-indexed" for short periods, then updated and re-introduced, so I insert the meta, then just remove it when I want the page found again. I find it simpler than keeping track of what's in or out of my robots.txt file.

Sasquatch

5:47 pm on Sep 25, 2002 (gmt 0)



How many links on this page full of links?

I'm no expert on it, but I doubt that google would consider one page a "link farm". Links pages are a net tradition, and since google thrives on links it looks to me like they go out of their way to find them. They spider my links pages a couple of times a day, I just wish they found my content pages as interesting.

I know of several non-spam pages with hundreds of links on them that have high PR. I would recommend that you just be careful about what sites you are linking to. Are any of them in a bad neighborhood?

bobmark

6:00 pm on Sep 25, 2002 (gmt 0)

10+ Year Member



ditto sasquatch!
One of the legitmately top ranking sites in my area is an index with tons of links out (and of course in) but also content on most pages.
I think to be flagged as a link farm you would have to be like a Zeus Themesite thing with endless links pages and virtually zero additional page content.