Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Google PageRank in cgi-bin

still showing page rank in cgi-bin

         

grant

4:46 am on Jul 20, 2005 (gmt 0)

10+ Year Member



About 8 months ago I realized that a script in my cgi-bin (messageboard) had PageRank. Intending to allow people to post links to their site freely, but also wanting to protect my site from link spammers, I blocked robots from the cgi-bin with robots.txt.

Several months later, the cgi-bin was still showing PageRank, and a search for the URL indicated that, indeed, the URL was still in Google's index.

Since then, I instructed our engineer to additionally employ .htaccess and meta noindex to make it very clear to Google that we do not want the cgi-bin spidered.

After the recent Google PR update, I noticed:

1. Areas of my sites that had ugly query strings that were addressed with mod rewrites now had PR
2. The cgi bin still had PR and was in the index!

I tried to check the last time the googlebot had spidered the cgi-bin, but was dismayed to realize our log files had reached their 2 gig limit months ago and were not flushed by our lazy engineers, so I cannot say WHEN Google last visited. It could be possible the bot hasn't visited in ages and the cgi-bin is just a historic artifact in the index.

Any ideas?