homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

Google is Deepcrawling
Is it for the next update or just checking links added in August?

 8:36 am on Aug 22, 2002 (gmt 0)

Google is deep crawling. Is it already crawling for the next update or just rechecking before the August update is effective?

By the way, in www2 I can see that my main keyword jumped from 31 to 11. Thank you everyone for sharing your knowledge here. I do not know where would I be without this forum.



 8:44 am on Aug 22, 2002 (gmt 0)

I've just seen her deepcrawling in yesterday's logs. She's fast this month. IMO, it's already for the next update.


 8:45 am on Aug 22, 2002 (gmt 0)

It's crawling for the next update but everyone still has enough time to put new content up till around...ooohhh, the 6-8th of next month.


 9:14 am on Aug 22, 2002 (gmt 0)

I guess only Google knows for sure, but I have to assume it is for the next update. There is no way Google could spider *all* the sites in the index to do rechecking. They are just gathering data for the next index.


 5:29 pm on Aug 22, 2002 (gmt 0)

I've had no clear spidering pattern across the sites I'm watching closely. One site PR5 has had a pretty thorough crawl starting just before the dance. Another PR6 has had just the home page hit about the same time. Another PR4 hasn't had a hit from GB since July 30.

I'm assuming that this is for the next index & she's just not gotten to the PR4 site yet. I've heard milk & cookies can help. Time to set it out.



 6:03 pm on Aug 22, 2002 (gmt 0)

If there's a month-to-month pattern, I can't detect it. For example one site was spidered between June 27-30 and the results appeared in the July update. The next deep crawl was the first week in August which I think is what will appear in the current update so my assumption is anytime from the 3rd week of the month to the start of the second week of the next.


 6:05 pm on Aug 22, 2002 (gmt 0)

HI all

This is all new to me but, I have launched a site a few weeks ago, it initially had a pr of 1, the whole site is now visable and my pr has gone up to 3.

I am most pleased, thanks to everyone who posts on this forum I am beginning to make sense of it all now



 6:42 pm on Aug 22, 2002 (gmt 0)

you guys make me wish i had a creditcard, so i could donate.


 7:53 pm on Aug 22, 2002 (gmt 0)

ok newbie question--

How can you tell when Google (any spider for that matter) is crawling? How do you check the logs for speed etc?



 8:02 pm on Aug 22, 2002 (gmt 0)

If you don't have good stats - and can use cgi - I would recommend you use this program:


You can learn A LOT by using this program.

Check out the view demo to see what it does.


 8:17 pm on Aug 22, 2002 (gmt 0)

I agree Chris_R
I use axs on key pages of my site to augment my logs. I also think that FDSE from the same source is a great way to acquire an understanding of how search engines work just by setting it up and using it on a site for visitor site search.


 1:13 am on Aug 23, 2002 (gmt 0)

How would you compare AXS with Webaliser stats package?

My host has this set up on our control panel as well as Analog.

Webaliser gives most popular entry and exit pages.
Most popular pages.
Top referrers by URL.
daily hourly and monthly stats for hits, pages, visits, kbs, etc.

Then Analog allows me to check which bots crawled my site each month and how many times - but does not show WHEN.


 1:22 am on Aug 23, 2002 (gmt 0)

Another agree here with Chris_R. I like it because it is showing page views only. Strips out the .css, .js, .jpg, etc stuff. I'm only interested in where a visitor came from and where he/she's going.

And it's really simple to find the googlebot. Just enter 216.239. or googlebot in the filter string and you'll see every hit.

As far as learning a lot: I learned A LOT of Perl and CGI just by looking at the two scripts. I've done a lot of manual editing of it to handle the referer strings (it's not too up to date in that area)

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved