Msg#: 4489670 posted 5:46 am on Aug 30, 2012 (gmt 0)
I have some pdf content that was allowed to crawled by robots, and then after I changed it to premium feature. Now its already indexed by google and even I dont want google to stop it indexing but as I have set robots.txt for those pdf files they are showing the result in google search like:
A description for this result is not available because of this site's robots.txt
So My question is :
Can I set any relevant meta description for these PDF files?
the TLDR version is you should set the meta title properly and use text rather than images. you will not see any type of meta description unless you remove the robots crawling exclusion for the pdfs. typically i see the first part of the text in the document as the description. i would suggest looking for cases where pdf metadata is used in snippets by doing some research with a filetype:pdf Google Search: http://www.google.com/search?q=filetype%3Apdf [google.com]
Msg#: 4489670 posted 4:30 pm on Dec 12, 2012 (gmt 0)
I wasn't sure if your were interested in setting meta information about the PDF or if you wanted something that allowed you control of a robot similar to the meta robots tag for html. If its the latter, then you may want to consider x-robots-tag http headers:
Msg#: 4489670 posted 11:00 pm on Dec 12, 2012 (gmt 0)
after re-reading the OP i think cpollett has the answer you were looking for but you must also allow crawling by removed those pdf url patterns from robots.txt so the bot sees the X-Robots-Tag headers.