Hi everyone. My website has less than one hundred pages of content but Google has indexed 1,500 pages. Most of it is junk pages associated with comments, feeds, tags, etc. This is our robots.txt: User-agent: *
Disallow: /admin
Disallow: /node/add
Disallow: /aggregator
User-agent: ia_archiver
Disallow: /
User-agent: Googlebot
Disallow: /topic
Disallow: /user/register
Disallow: /user/login
Disallow: /comment
Disallow: /*/feed$
Here are the pages that I think *should* be excluded by isn't:
www.example.com/topic/heating
www.example.com/comment/reply/14
www.example.com/wine-2/feed
Moreover, here are some pages I would like to exclude, but isn't sure how to do it using the robots txt:
www.example.com/blog?page=11
www.example.com/home?page=13
The robots text has been up for a month and a half now, should I wait longer for it to take effect? Thanks in advance for any help!