homepage Welcome to WebmasterWorld Guest from 54.167.41.199
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Need google to stop crawling and indexing a particular directory
how to set up an .htaccess file for this dir?
Jessica




msg:3198787
 11:54 am on Dec 24, 2006 (gmt 0)

Please help me out to stop google from crawling a particular directory and all it's contents.

I have many incoming links to this directory and pages. how can i channel the PageRank to my main page? I need to set up some kind of redirect in .htaccess?

 

WiseWebDude




msg:3200294
 10:13 pm on Dec 26, 2006 (gmt 0)

No you need to create a robots.txt file in the root of your site:

www.example.com/robots.txt

User-agent: *
Disallow: /foldername/

a simple text file with nothing but:

# /robots.txt file for http://www.example.com/

User-agent: *
Disallow: /foldername/

[edited by: WiseWebDude at 10:14 pm (utc) on Dec. 26, 2006]

Span




msg:3200316
 10:37 pm on Dec 26, 2006 (gmt 0)

Hi,

disallowing Google to crawl a directory is not the solution for your "PR4 on all internal pages, but PR0 on the index page [webmasterworld.com]" problem.

Try to understand that first before taking any wild actions. This won't channel any PR to your main page.

Jessica




msg:3200613
 7:29 am on Dec 27, 2006 (gmt 0)

Thats the other site Span.

Jessica




msg:3200614
 7:30 am on Dec 27, 2006 (gmt 0)

anyway, i did a 301 javascript redirect.

abacuss




msg:3200618
 7:45 am on Dec 27, 2006 (gmt 0)

Javascript Redirection? If it is a Permanent Redirection through 301 then its ok otherwise you run the risk of being termed as a spammy site because of sneaky redirects.

Jessica




msg:3200623
 8:02 am on Dec 27, 2006 (gmt 0)

whats a "Permanent Redirection" ?

the pages are still there, i just redirect them to the main page with JS.

OutdoorMan




msg:3201179
 9:03 pm on Dec 27, 2006 (gmt 0)

whats a "Permanent Redirection"?
the pages are still there, i just redirect them to the main page with JS.

You'll need to use server side scripting (php, asp, asp.net or so) to make a permanent redirect (a socalled '301 Permanently Moved' header status code).

This cannot be done by javascripting.

Jessica




msg:3201191
 9:21 pm on Dec 27, 2006 (gmt 0)

i have a lot of external backlinks pointing to those pages.
i dont want to lose the PR, i want to channel it back to the main page by redirect.

which redirect would work for that?

WiseWebDude




msg:3202165
 9:02 pm on Dec 28, 2006 (gmt 0)

301 redirect via .htaccess file. Do NOT do javascript OR "meta refresh" redirects.

one line in .htaccess:

Redirect permanent /olddirectory/oldfile.html http://www.example.com/newdirectory/newfile.html

of course, put your info in for the actual site, LOL.

[edited by: WiseWebDude at 9:04 pm (utc) on Dec. 28, 2006]

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved