Welcome to WebmasterWorld Guest from 23.21.38.201

Forum Moderators: goodroi

Robots text to block all but index page of a forum

   
10:41 pm on Dec 21, 2011 (gmt 0)



I'm trying to generate an robot text file to disallow all SEs on all pages of forum except the index page.

So I want to allow it to index:

mysite.com/forum/


But anything else in the forum directory and subdirectories, I don't want indexed.

mysite.com/forum/$anythingelse$

I remember a command with a question mark in it that worked in the past. But I can't find any guidance on that.


Does anyone know how to do that?
12:25 am on Dec 22, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



What directly follows /forum/ in the URLs to be blocked?

Is it any letter from A to Z or selected letters or certain digits?
1:06 am on Dec 22, 2011 (gmt 0)



All are letters, no numbers at all.

Thanks for the help.
2:53 am on Dec 22, 2011 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



You could do the following
Disallow: /forum/a
Disallow: /forum/b
Disallow: /forum/c
etc.
Disallow: /forum/z


The question mark and other wild cards are not supported by all bots.
7:35 am on Dec 22, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Is it any letter from A to Z or selected letters?

Are they all the same case?
11:31 am on Dec 22, 2011 (gmt 0)



It's a number of different letters maybe 10.

They are all lower case.

Thanks a lot guys.
11:41 am on Dec 22, 2011 (gmt 0)

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



Allow forum/index.html
Disallow forum/

Works for me in real life.

mysite.com/ is not required

Order is important.
12:34 pm on Dec 22, 2011 (gmt 0)



Thanks tangor. You guys rock.
 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month