homepage Welcome to WebmasterWorld Guest from 54.205.189.156
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Googlebot doesn't crawl internal pages since 4/1
could it be robots.txt?
cwizard




msg:715486
 4:56 pm on Apr 12, 2005 (gmt 0)

My site has 144 pages that I created/published with FrontPage 2000. 7 (including my home page) are keyword-optimized and I'm tracking them in the SERPs. All 7 pages (and many others) have been edited and published daily for at least the last 2 weeks. My home page is optimized for 3 keywords and doing fine. Apparently none of the 6 internal pages I'm tracking have been crawled since 4/1/05 (the most competitive fell off the SERPs since then until yesterday when it limped back to the 17th page). I assume this is so since Google's last line in the INTERNAL PAGES SERPs is their URL followed by a "Cached" link, which when clicked shows that page "as retrieved on April 1, 2005 when Google crawled the Web." My home page, however, shows its URL FOLLOWED BY THE DATE OF THE LAST CRAWL, which Google has updated daily. (Remember, I've daily edited and published all 7 pages and many others over the last 2 weeks). Grasping at straws and because I don't understand HTML that well, is it possible it's because the "Head" section on all pages ends with the two lines:
<meta name="robots" content="index,follow">
<meta name="GOOGLEBOT" content="INDEX,FOLLOW">

Any thoughts?

 

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved