Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- And Now Google's Doing It. JS Stats Show GoogleBot


keyplyr - 4:45 am on May 15, 2011 (gmt 0)



If yours are not being indexed it's probably because there aren't enough links to them for them to be indexed without knowing the content anyway, but it's got nothing to do with the robots.txt block, because those pages will be indexed as URL only ... Check out the Google forum if you haven't ... This is one of the biggest points of confusion ... If you have seen the Google forum, then, uh, you should know locations blocked in the robots.txt are easily (and often) indexed.

I never said I was having problems getting links indexed. I'm not the one having the problems, this is your thread. I will however be sure to remember not offer any insight to you again.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4312058.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com