Forum Moderators: goodroi

Message Too Old, No Replies

directory structure question

         

union_jack

9:45 am on Jan 19, 2007 (gmt 0)

10+ Year Member



I want to block some duplicate pages and I am slightly confused with directory issues with robot.txt.

My domain structure is
domainname.com/area/brochures/name.htm

If i wish to block these pages, is this correct?

User-Agent: *
Disallow: /area/brochures/

or can I use

User-Agent: *
Disallow: /brochures/

I have about 50 areas so if the first example is required will this be a problem?

thanks 4 any help?

concepts99

4:04 pm on Jan 21, 2007 (gmt 0)

10+ Year Member



Can anyone reply on this, I have the same question.

union_jack

3:33 pm on Jan 22, 2007 (gmt 0)

10+ Year Member



mmm, guess it was a stupid question as nobody answered it lol

beesticles

5:41 pm on Jan 22, 2007 (gmt 0)

10+ Year Member



I don't think this is a stupid question! Unfortunately I think you're going to have to do it the hardway, e.g.

User-Agent: *
Disallow: /area1/brochures/
Disallow: /area2/brochures/
Disallow: /area3/brochures/
etc.

I don't think that the wildcard syntax is that sophisticated with robots.txt.

union_jack

7:43 pm on Jan 22, 2007 (gmt 0)

10+ Year Member



thank 4 that, i was looking for a short cut, but I guess better to be safe than sorry :-)