homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

Googlebot - Sequentially Spidering?

5+ Year Member

Msg#: 28263 posted 6:58 pm on Feb 23, 2005 (gmt 0)

Just curious if anyone is seeing the following phenomenon:

A single Google Freshbot ( spiders my site daily taking thousands of pages however, the the pages are spidered based on the length of the URI (ie. [example.com...] where the keyword-phrase is the URI.

This particular bot is spidering the shorter URI's FIRST, then sequentially coming back for the longer mulit-phrase URI's. ie.


Has anyone else seen this behavior?

[edited by: ciml at 7:58 pm (utc) on Feb. 23, 2005]
[edit reason] Examplified [/edit]



10+ Year Member

Msg#: 28263 posted 8:05 pm on Feb 24, 2005 (gmt 0)

yeah, i've seen that sometimes ... i think there have been some threads discussing this before.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved