Forum Moderators: open

Message Too Old, No Replies

Disallowing google bot from a specific directory

         

mack

7:09 am on Oct 14, 2002 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



If you disallow google form crawling a specific directory on your site will all the pages within that directory apear PR0 or will the toolbar make a guestimate of PR from the Page rank at root?

fathom

7:26 am on Oct 14, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Pages must be indexed in Google to have PageRank. Disallowing googlebot from a directory will provide a Grey Bar.

In effect, according to Google the directory (and all its pages) would not exist... at least until you allow the bot to crawl.

mack

8:08 am on Oct 14, 2002 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Thanks for your reply. The reason i was thinking thie was, when you upload new pages they always deam to have some pagerank, even although they have nevr been crawled?is it possible it will index the link to it, but not index the page it's self and this may be how it gets PR?

fathom

8:32 am on Oct 14, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



It is true that the bar will take a guessament based on your link structure and hierarchy but this really isn't PageRank.

Once googlebot comes around to crawl and finds out you do not want these ages indexed (the directory) they will be Greyed out.

Use a robot tag "noindex, follow" and also place an -- "If Modified Since" tag.

I still doubt any PR will be passed, again since not indexed, these pages have no PR.

mack

10:13 pm on Oct 14, 2002 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Thank for clearing that up.

I asumed the estimate remained, but as you pointed out once it trieds to crawk and is not allowed the pages will rank grey :D