seochristine - 12:05 pm on Oct 18, 2005 (gmt 0) User-agent: GoogleBot I presume the format is not incorrect.
I am new to the webmasterworld.
I have been observing a strange behavior in Google, I have disallowed a few of my files in the robots.txt document so that they dont get crawled by the spider. But the stunning part is that the files are getting indexed, when I perform the search using "site:mydomain.com" in Google, all those files are displayed in the SERP.
The specific format I am using in the robots.txt file is as follows:
Any feedback in this regard will be very helpful.
I presume the format is not incorrect.