Welcome to WebmasterWorld Guest from 188.8.131.52 , register , free tools , login , search , subscribe , help , library , announcements , recent posts , open posts Subscribe to WebmasterWorld
Google showing site despite blocking in robots.txt The page is not cached though ecmedia msg:3509872 4:20 pm on Nov 20, 2007 (gmt 0) I do not want Google to index a recently launched website and followed its guidelines. However, in the search results I see a listing like this: www.domain.com Similar pages There is no title or description or cache or file size. Is this what I should have expected?
pageoneresults msg:3509875 4:25 pm on Nov 20, 2007 (gmt 0)
Yes, this is normal behavior. It is referred to as a URI Only Listing and occurs in this scenario. No other pages from the site will show. Just the root level reference.
The Contractor msg:3509880 4:28 pm on Nov 20, 2007 (gmt 0)
yep, as long as they have already indexed the page(s). If you do not want the pages indexed you need to put one of the following in between the page(s) head tags:
<meta name="robots" content="none">
<meta name="robots" content="noindex,nofollow">
and then change your robots.txt to re-allow them to get to the page.