homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

Robots.txt on phpbb forums
it worked! thanks for the tip

WebmasterWorld Senior Member 10+ Year Member

Msg#: 24 posted 1:52 pm on Jul 29, 2003 (gmt 0)

I have a site that's running a phpbb forum, and google seems to index specific number of pages from the site no matter what I do. The problem is that many of the pages google was indexing were worthless such as the login for private messages. Following the advice of this post...
...I created my own robots.txt file:
User-agent: *
Disallow: /groupcp.php
Disallow: /login.php
Disallow: /modcp.php
Disallow: /privmsg.php
Disallow: /search.php
It worked like a champ and now Google is getting at the good stuff and leaving the spider poop alone.
Thanks for the tip!


Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved