Forum Moderators: goodroi

Message Too Old, No Replies

Wildcards in robots.txt

         

madmatt69

11:40 pm on Mar 30, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Googlebot's being a punk and grabbing files it shouldn't.

Basically, I have a forum and I've disallowed pages like posting.php, profile.php etc. The problem is that when someone views those pages it often shows up as "posting.php?quote=?randomvariable"

And google treats them as individual pages and indexes them.

Right now I have:
Disallow: /forums/profile.php

For example - Can I make that profile.php* or something that means ANY file with profile.php in it will NOT be spidered?

Any help is much appreciated!

madmatt69

12:52 am on Mar 31, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If I used Disallow: /forums/profile.php$ would that make a difference?