FAQ: Everflux - mid-month Google spidering and minor updates - Sept. 2002

The minor activity in between major Google updates

[Marcia]

There's one major Google update per month, which starts approximately the 20th to the 28th of each month and continues for several days. The new data migrates to the partner sites within several days.

See the Google Update History [webmasterworld.com] for details.

There is a major crawl at this time, with some reporting full crawls simultaeously with the start of the update or just prior, while others are not crawled until after the update is complete.

For several months there's been spidering of sites going on all month, even in between the major updates, but this only represents a small percentage of sites and while it may or may not affect ranking changes, it doesn't affect the entire index.

There was a lengthy discussion of what's been called The Everflux Effect early in July:

[webmasterworld.com...]

We've been having a lot of threads starting with questions about this all month long, which makes it hard to find all the relevant information related to it. Also, the volume of posts is making it harder to find other information relevant to Google.

Let's confine discussion of the mid-month spidering and minor update pattern to one thread, to help everyone find the information they need more easily.

[ikbenhet1]

so that the www2 result is diffrent from the www result means nothing for now? And this Next Crawl is fully reset the www2 and rerank again?

[razael]

For a week my site was indexed by google and by searching some phrase from the site it would appear, but I've checked today and it is nowhere to be found.
I'd appreciate any info and advice on the subject, I haven't done anything wrong but the site is hosted on virtual host so maybe someone else did something wrong and the whole IP on which all the domains are hosted got banned?

Thanks

[edited by: Marcia at 3:13 am (utc) on Sep. 21, 2002]
[edit reason] url removed [/edit]

[Marcia]

Welcome to WebmasterWorld, razael.

This is nothing to worry about, it's normal for a newly added site to appear briefly and disappear mid-month. If you read back through this thread, you'll see that there are changes going on all month.

Good luck with the coming update, and with your new site.

[razael]

Thanks Marcia :p
I wrote the post in panic, I usually read all the faq and info on the subject before posting but..it is 5.30 am here :) so I had to do something.
Anyways the site was indexed on 11 or 13 of September but only the Index page and nothing more, and now I see it is nowhere to be found

Cheers
Blagoja

[Big_Balou]

Just started to see G*bot again. Last visited concurrently with last months update.

Anyone else seeing renewed activity?

[coconutz]

Just the usual daily crawl.

[Mills_zaboy]

Googlebot is gone wild

[zeus]

This was just a wierd update late Aug., I have had 7 new dates on my cache, Sites indexed is changing every hour and my page mooves from page 1 on some keywords to page 100 every 4 day and I have a PR0.

All this happen from the last update, so it has been a wierd update for me.

zeus

[flex55]

2 days ago Googlebot hit my site for the first time, and I was the happiest man ever... :-)
It didn't really crawl too dip, I got just 2 hits, but I made it to www2 (and surprisingly also to www).
However, yesterday (only 1 day after the first visit) Googlebot hit again - just one hit, which was enough to drop me off www and www2.
I can't even try to describe how desperate I got.
I din't change anything at all in the site. I'm 100% sure it was up and running. I just dont get it...

Does anyone have any clue why would that happen? Did anyone experience something similar (ie, getting in and out of the index within 2 days ;-)? All feedback is highly appreciated!

[electro]

Unless you have some kind of a spammy/ dodgy site, it's probably just a blip, if you have some inbound links, it's unlikely that you won't get into the update, which should be any day now. Hold tight.

[zero6]

flex,

My new site is doing just that. It drops in and out everyday, but it's also getting hit everyday by ggb and when it is listed it's fresh.

I wouldn't worry too much, if its a new site or just never been picked up before then more than likely it will settle in a bit after the update... whenever that happens.... ;)

z6

[flex55]

Ahh. Huge reliefe ;-)
We're back in the index.
Man. You can easily get a breakdown from these things :-)

[Helpmebe1]

Good Post Marcia.. but question...

How long does google deepcrawl your site for.. as I mentioned.. I am a yahoo store.. I dont know if I should even bother trying to update for the dance at the end of this month... should I? Or is it probable that google already crawled and left and my efforts right now would be wasted as it does not matter if i then update today or two weeks from now?

[Marcia]

Helpmebe, it's best not to make major changes in the middle of updates/crawls, but sometimes last minute alterations or additions do get picked up.

Basically, update a site with customers and sales in mind. If there's traffic and you have more merchandise to put up and sell, every day delayed can be lost revenue. You just keep your navigation consistent.

For changes affecting site design my personal opinion is that mid-month is better, to make the next EOM crawl and get in the following round - which would be 45 days off. I generally figure that what goes up prior to update (last week of the month) will get in the following month.

Sites pop in and out now, but it's unnecessary stress to expect anything until the update following the end of the current month.

[Helpmebe1]

Marcia,
You say not to make changes in the middle of a crawl? Is that for your own personal knowledge as if to something worked better one way or another? I am not sure why you say dont make them during a crawl. Well its not major changes, just additional product going up... and true, the longer you wait is another day you cant sell it ... They will be up over the next two weeks or so giving I can find the time to get it all done.. just was wondering do I cram this weekend to make the deadline or is it not worth it? Noone seems to know exactly when google crawls nowadays, where I could of missed it or maybe didnt? Anyones guess?