Forum Moderators: phranque

Message Too Old, No Replies

I need to block my PDFs from spidering

Is this even possible?

         

annej

12:49 am on Mar 19, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I have some PDFs that are a part of some patterns I have online. I use them for the exact sizing I need. They are secondary to the articles and I don't want people finding them instead of the basic information. I can't imagine how to put a noindex on them though. Can anyone help me on this?

jomaxx

1:11 am on Mar 19, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



List them individually in the robots.txt file. Or (better solution) put them in a separate directory that is blocked via the robots.txt file.

jdMorgan

3:34 am on Mar 19, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If you're still stuck, Googlebot and Slurp will accept

Disallow: /*.pdf$

however, you cannot count on any other robots to recognize that syntax unless they explicitly state that they will in their webmaster info page, so it's best used in individual-robot-specific records in robots.txt

Jim

annej

9:42 pm on Mar 22, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thanks to both of you.

The separate directory sounds like the easiest way to go.