closed
msg:1526473 | 5:31 am on Mar 12, 2004 (gmt 0) |
Maybe with a robot that was set up to use patterns like that. But normal robots won't understand it.
|
DoppyNL
msg:1526474 | 7:57 am on Mar 12, 2004 (gmt 0) |
Then what would be the best way to accomplish this. url: /*1/file/*2/ Taking into account that: - "*1" can be practicly anything and a lot of different values are possible
- "*2" will be short, allthough it can be a lot of different values. What would be the best way to prevent indexing by bots?
I can't use the meta-tag, as these are images and other files (non-html).
|
closed
msg:1526475 | 5:49 am on Mar 13, 2004 (gmt 0) |
Well, if you're going to disallow everything within file, you can just shorten it to /*1/file/. There's no pretty way to deal with the different values of *1, though. I just write out each case. If you're gonna end up with lots of lines of Disallows, you could probably write a script that will traverse your directory structure and output a robots.txt for you.
|
DoppyNL
msg:1526476 | 8:51 am on Mar 13, 2004 (gmt 0) |
hmmm, too bad Something that needs improvement if you ask me :) oh well, I'll live with it. I'll probably make a script that will make the robots.txt on the fly when it's requested; simple check in the database will give me the answer on that :)
As *1 is subject to change so often that it's to much work to do that on every change. tnx for the input!
|
closed
msg:1526477 | 5:36 pm on Mar 13, 2004 (gmt 0) |
Yeah, it's a pain in the you know what. :-) You're welcome, DoppyNL.
|
|
