Welcome to WebmasterWorld Guest from

Forum Moderators: Robert Charlton & andy langton & goodroi

Message Too Old, No Replies

Googlebot not obeying noindex, nofollow META Robots Tags.

5:16 pm on Jun 30, 2006 (gmt 0)

New User

10+ Year Member

joined:May 16, 2006
votes: 0

I recently deployed a new site and Googlebot has not obeyed the noindex , nofollow META Robots Tags. As a consequence Google has indexed a lot of content on my site that I did not want indexed. How can I get Google to drop those pages and ignore them in the future?
9:18 am on July 3, 2006 (gmt 0)

New User

10+ Year Member

joined:June 28, 2006
votes: 0

Remove content from Googles index [google.com]

Where abouts have you placed your robot.txt file and what is the exact contents of the file.

You may havesimply made a mistake.

9:41 am on July 3, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 31, 2004
votes: 0


Frankly im finding on a number of sites that google is doing just about what it likes.

Not just ignoring no index, no follow robot tags etc but also its now reading files within java script etc.

Meanwhile googles attitude is kind of "Well if you dont want us to crawl your site we will take it out the index"

Bottom line is imo that you will have to either put up with the abuse of google ransacking everything on your pages OR get your site removed from its search.

Not much of an option!

10:45 am on July 3, 2006 (gmt 0)

New User

10+ Year Member

joined:May 17, 2005
votes: 0

G has been ignoring robots.txt and on page directives for a while now.

Best option, use rewrite to block the folder the pages are in


Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members