Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

robots.txt on phpbb forum

4:55 pm on Dec 6, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:May 20, 2003
votes: 0

Someone made a mod that allows all spiders to spider a phpBB forum. I was wondering if there would be a way to prevent them from spidering some of the sections of the forum because they aren't relevent to the site content. So, say these are the sections I have:


I want the first section spidered, but not the second or third. Is this possible?


4:06 pm on Dec 12, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:July 24, 2002
votes: 0

hi jennifer,

as far as i know you can simply specify the forum you wish not to be crawled. e.g.

User-agent: *
Disallow: /phpbb/viewforum.php?f=18

this is perfectly valid and it also complies with googles robots.txt info page [google.com]

to be on the safe side though this post bumps yours to the top anyway ;-)



hi jennifer, i have just added a bit of php code to two of the phpbb files which adds a <meta name="robots" content="noindex,nofollow"> depending on the forum_id

sticky me if you wish for the code


Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members