Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Which Proxy headers do you capture?
Clark - 11:11 pm on Oct 27, 2010 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4222008.htm
Thanks Jim. I coded it to grab everything. After I launch and give it some time to populate, I'll report back with my findings too.
Brought to you by WebmasterWorld: http://www.webmasterworld.com