Need google to stop crawling and indexing a particular directory

how to set up an .htaccess file for this dir?

[Jessica]

Please help me out to stop google from crawling a particular directory and all it's contents.

I have many incoming links to this directory and pages. how can i channel the PageRank to my main page? I need to set up some kind of redirect in .htaccess?

[WiseWebDude]

No you need to create a robots.txt file in the root of your site:

www.example.com/robots.txt

User-agent: *
Disallow: /foldername/

a simple text file with nothing but:

# /robots.txt file for http://www.example.com/

User-agent: *
Disallow: /foldername/

[edited by: WiseWebDude at 10:14 pm (utc) on Dec. 26, 2006]

[Span]

Hi,

disallowing Google to crawl a directory is not the solution for your "PR4 on all internal pages, but PR0 on the index page [webmasterworld.com]" problem.

Try to understand that first before taking any wild actions. This won't channel any PR to your main page.

[Jessica]

Thats the other site Span.

[Jessica]

anyway, i did a 301 javascript redirect.

[abacuss]

Javascript Redirection? If it is a Permanent Redirection through 301 then its ok otherwise you run the risk of being termed as a spammy site because of sneaky redirects.

[Jessica]

whats a "Permanent Redirection" ?

the pages are still there, i just redirect them to the main page with JS.

[OutdoorMan]

whats a "Permanent Redirection"?
the pages are still there, i just redirect them to the main page with JS.

You'll need to use server side scripting (php, asp, asp.net or so) to make a permanent redirect (a socalled '301 Permanently Moved' header status code).

This cannot be done by javascripting.

[Jessica]

i have a lot of external backlinks pointing to those pages.
i dont want to lose the PR, i want to channel it back to the main page by redirect.

which redirect would work for that?

[WiseWebDude]

301 redirect via .htaccess file. Do NOT do javascript OR "meta refresh" redirects.

one line in .htaccess:

Redirect permanent /olddirectory/oldfile.html http://www.example.com/newdirectory/newfile.html

of course, put your info in for the actual site, LOL.

[edited by: WiseWebDude at 9:04 pm (utc) on Dec. 28, 2006]