I find it puzzling that this once active forum makes it so difficult for newcomers to learn! (rhetorical).
Is it so easy to grasp that we all began with 1+1=2 and not geometrical equations?
There are long time participants (fifteen years or more) that seem to add such heavy warnings of caution for that one good guy (bot) that offers an advantage, that the newcomer is faced with the impossible task of accepting defeat before they've even begun.
born2,
Here's some commonly misused UA's and are only meant for a guideline.
You must explore each one with your own raw logs and determine for yourself whether using each word is detrimental or beneficial. client crawl curl client download
Fetch grab Java Library libww
Nutch parser reaper Retrieve spider
These threads (Close to perfect htaccess) are more than fifteen years old. They are very long, repetitious and participants copy and pasted both UA's and syntax from unknown other sources, even if the syntax was incorrect (there are many, many errors).
With all the above in mind, the threads are still useful guidelines. Once again,
you must explore each one with your own raw logs and determine for yourself whether using each word is detrimental or beneficial. It is also imperative to note that after each and every modification (upload) of you htaccess that you confirm your site (s) are still working (not generating a 503).
[
webmasterworld.com...]
[
webmasterworld.com...]
[
webmasterworld.com...]
[
webmasterworld.com...]
[
webmasterworld.com...]
As others have explained to you, there is no one-size-fits-all. Copy and pasting what one (or more) webmaster uses is not a sound practice, (at least not without exploring all the detriments to your site (s)). You need to analyze each addition and determine the overall benefit as related to your websites goals (only you know that).