Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Google showing site despite blocking in robots.txt

The page is not cached though



4:20 pm on Nov 20, 2007 (gmt 0)

5+ Year Member

I do not want Google to index a recently launched website and followed its guidelines. However, in the search results I see a listing like this:
Similar pages
There is no title or description or cache or file size.
Is this what I should have expected?


4:25 pm on Nov 20, 2007 (gmt 0)

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Yes, this is normal behavior. It is referred to as a URI Only Listing and occurs in this scenario. No other pages from the site will show. Just the root level reference.

The Contractor

4:28 pm on Nov 20, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

yep, as long as they have already indexed the page(s). If you do not want the pages indexed you need to put one of the following in between the page(s) head tags:

<meta name="robots" content="none">


<meta name="robots" content="noindex,nofollow">

and then change your robots.txt to re-allow them to get to the page.


Featured Threads

Hot Threads This Week

Hot Threads This Month