Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- 'artviper(tm)


motorhaven - 8:43 pm on Dec 13, 2012 (gmt 0)


Yes, often there is, but I try when I can identify it to block around the user IPs because my user content driven sites (forums) depend on them. Often my most prolific posters are what we commonly call "post whores", who post all day long from their job. Their posts tend to drive many responses... hence I let them through.

Only legit users I block are those coming in through anonymous proxies because the same proxies also tend to bring comment spammers.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4526792.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com