Welcome to WebmasterWorld Guest from 54.224.5.186

Forum Moderators: Robert Charlton & andy langton & goodroi

Message Too Old, No Replies

Need google to stop crawling and indexing a particular directory

how to set up an .htaccess file for this dir?

     
11:54 am on Dec 24, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:July 3, 2003
posts:145
votes: 1


Please help me out to stop google from crawling a particular directory and all it's contents.

I have many incoming links to this directory and pages. how can i channel the PageRank to my main page? I need to set up some kind of redirect in .htaccess?

10:13 pm on Dec 26, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:June 8, 2006
posts:556
votes: 0


No you need to create a robots.txt file in the root of your site:

www.example.com/robots.txt

User-agent: *
Disallow: /foldername/

a simple text file with nothing but:

# /robots.txt file for http://www.example.com/

User-agent: *
Disallow: /foldername/

[edited by: WiseWebDude at 10:14 pm (utc) on Dec. 26, 2006]

10:37 pm on Dec 26, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 30, 2004
posts:712
votes: 0


Hi,

disallowing Google to crawl a directory is not the solution for your "PR4 on all internal pages, but PR0 on the index page [webmasterworld.com]" problem.

Try to understand that first before taking any wild actions. This won't channel any PR to your main page.

7:29 am on Dec 27, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:July 3, 2003
posts:145
votes: 1


Thats the other site Span.
7:30 am on Dec 27, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:July 3, 2003
posts:145
votes: 1


anyway, i did a 301 javascript redirect.
7:45 am on Dec 27, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 29, 2005
posts:140
votes: 0


Javascript Redirection? If it is a Permanent Redirection through 301 then its ok otherwise you run the risk of being termed as a spammy site because of sneaky redirects.
8:02 am on Dec 27, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:July 3, 2003
posts:145
votes: 1


whats a "Permanent Redirection" ?

the pages are still there, i just redirect them to the main page with JS.

9:03 pm on Dec 27, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Sept 18, 2006
posts:197
votes: 0


whats a "Permanent Redirection"?
the pages are still there, i just redirect them to the main page with JS.

You'll need to use server side scripting (php, asp, asp.net or so) to make a permanent redirect (a socalled '301 Permanently Moved' header status code).

This cannot be done by javascripting.

9:21 pm on Dec 27, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:July 3, 2003
posts:145
votes: 1


i have a lot of external backlinks pointing to those pages.
i dont want to lose the PR, i want to channel it back to the main page by redirect.

which redirect would work for that?

9:02 pm on Dec 28, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:June 8, 2006
posts:556
votes: 0


301 redirect via .htaccess file. Do NOT do javascript OR "meta refresh" redirects.

one line in .htaccess:

Redirect permanent /olddirectory/oldfile.html http://www.example.com/newdirectory/newfile.html

of course, put your info in for the actual site, LOL.

[edited by: WiseWebDude at 9:04 pm (utc) on Dec. 28, 2006]

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members