Page is a not externally linkable
- Search Engines
-- Ask - Teoma
---- entire site "mis"-crawled with appended % 20 codes.


jdMorgan - 7:03 am on Dec 29, 2007 (gmt 0)


Oops! I introduced some errors in the generalize-copy-and-paste, there. The first RewriteCond should read:

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([/0-9a-z._\-]*)[^/0-9a-z._\-\?\ ][^?\ ]*(\?[^\ ]*)?\ HTTP/ [NC]

Also, the most likely cause of this problem is search engine robots picking up on links posted in forums, where the poster or the forum software's auto-link routine has included the trailing space in the link.

You'll also see this happening with a trailing period on the requested URL when the person posting the link puts a period at the end of it -- as in, "For more info, see http://example.com/widget.html." However, in this case the period is not hex-encoded, because it is a valid character to include in a URL, unlike a space.

Jim


Thread source:: http://www.webmasterworld.com/ask_jeeves_teoma/3536339.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com