FAQ: Everflux - mid-month Google spidering and minor updates - Sept. 2002

The minor activity in between major Google updates

[Marcia]

There's one major Google update per month, which starts approximately the 20th to the 28th of each month and continues for several days. The new data migrates to the partner sites within several days.

See the Google Update History [webmasterworld.com] for details.

There is a major crawl at this time, with some reporting full crawls simultaeously with the start of the update or just prior, while others are not crawled until after the update is complete.

For several months there's been spidering of sites going on all month, even in between the major updates, but this only represents a small percentage of sites and while it may or may not affect ranking changes, it doesn't affect the entire index.

There was a lengthy discussion of what's been called The Everflux Effect early in July:

[webmasterworld.com...]

We've been having a lot of threads starting with questions about this all month long, which makes it hard to find all the relevant information related to it. Also, the volume of posts is making it harder to find other information relevant to Google.

Let's confine discussion of the mid-month spidering and minor update pattern to one thread, to help everyone find the information they need more easily.

[MikeMike]

Well this is a nice "minor update" for me.
In my search on www I get about 5,900,000 results with my site as #40+. In www3 I get 5,200,000 results and my site is #7.
I have been deep crawled yesterday and today, so to me this looks like a regular update.

[jen24815]

Hi,

I was wondering if the "everflux effect" affected all sites or just newly included ones - and if it ever settles down?

One of my sites was created around the 1st week of July and was included in the July update at #10 for my main phrase. It stayed that way w/o change until the August update, at which time it bounced between #6 and #7 for the same phrase during the dance and then around the end of August, dropped off the face of the planet. :-(

The strange thing is, during the dance, it was NOWHERE to be found in www3 and www2, but still was bouncing on 6 or 7 in www (up from it's pre-dance position).

Now it pops in and out of www at 6 or 7 about 2 or 3 times a day and then disappears the rest of the day, and on www2/3 it has moved to page 2 and is steadily climbing up.

What gives here? Is this everflux or just some weird "glitch" that I wish would happen on any search engine OTHER than Google since I REALLY like those brief shining moments at #6. ;-)

Thanks,

Jenny

[zeus]

jesus I got a new cache again this is the 4th time in 2 weeks, marked 4 sep., but still PR0

zeus

[ciml]

Welcome to WebmasterWorld, Ambiorix.
(Make sure to check out paynt's welcome post [webmasterworld.com])

I would not worry if you have multiple sites on one IP address, if you do things that Google don't like then using different IPs doesn't help in my experience.

Changing IP address is often a problem, you may disappear from Google for a month or even two.

I would be very surprised if Gooogle penalised you for mentioning Google too often. Of course linking to SEO sites (especially those penalised for link farming) is another matter.

[johnser]

ciml

How about if you've been ranked top for over a year for highly competitive KWs, are not doing anything spammy, yet you've just been dumped in the SERPs???

This is the situation for a company I know of with 20 sites. The only remotely bad thing they do as far as I can see is have them spread across the same 3-4 IPs. Nothing else.

Any clues? can stickymail you the url but its very standard
J

[lazyz]

Johnser - your post and this reply is off topic for this dicussion but "not doing anything spammy" is usally an indication you don't know how you are spammy... Meaning - if you read through the hundreds of messages in the Google area alone, you might discover something new - that you didn't know - and figure out how you are actually doing something spammy... Hopefully you will get my drift.

I've spammed. I didn't know how I was spamming until after my PR0 penalty. I've learned my lesson and to be on topic - I have several new domains with a cache date of September 8th... and I am showing high in the serps but history shows me that just because my site is caught up in the ever-flux it doesn't mean I will be in the update at the same position next month during the normal update.

[egomaniac]

After watching the watching my site bubble around in the everflux these past few weeks, I've come to a couple of conclusions:

1) Google now is using multiple indices that alternate all month long between updates. I believe that there are 2 or 3 of these indices.

2) Each of these indices is subject to minor but constant ongoing change between updates (what we've dubbed the everflux effect).

Conclusion #1 is very significant as it is a new wrinkle in how Google works. It has been going on for at least two update cycles from what I can tell.

I believe that this is the latest manifestation of Google's desire to have more frequent updates. It isn't really more frequent, but the results would appear to be more dynamic to the non-SEO-geek-observer. :)

[dvduval]

Rules of Everflux:

1. Google updates every 2-4 days
2. Each update usually includes pages from 2 dates. For example, this update includes Sept 9 and Sept 10th.
3. Pages older than 2-3 days lose the fresh date. For example, the pages that said Sept 8 no longer say anything.
4. Contrary to some comments I've seen, there are not any pages that get a daily update.
5. Placing a link on a page that is refreshed on every update causes the linked page to get updated most of the time.

[nutsandbolts]

Almost a totally different set of results for one competitive keyword this morning - yikes!

[Peggo]

Hi!

I'm new to this forum and want to say Hello to all!

I also noticed a wierd shift in search results today.

Any idea what's going on?

Peggo

[ciml]

Welcome to WebmasterWorld [webmasterworld.com], Peggo.

> Any idea what's going on?

These kinds of changes are normal nowadays. Part of it is the general shifting as a result of the 'Fresh' listings (a site search [searchengineworld.com] should explain), and the rest is mystery. It's the Everflux.

[dazz]

ve just seen lower PR on some sites......bt openworld is now 4 from 5........anyone else?

[Jane_Doe]

I just had a new page added to Google. It's the first time I've ever had a new page added between the monthly updates. Way cool! It came in at #6 for it's search term.

For the rest of the pages I've checked, I see minor ranking changes, i.e. up or down one or two positions.

[egomaniac]

>Almost a totally different set of results for one competitive keyword this morning - yikes!

This is what I am talking about. Google is alternating results between 2 or 3 different indexes. Do the same search over different times of the day, and over a few days and you'll see what I mean.

For a given search, you'll generally notice a pattern of 2 or 3 different total results, as well as different rankings.

This has been going on for at least 2 mos. now.

[Marcia]

I've seen minor fluctuations of a spot or two, especially where there are sites with the every other day refresh. But more than that, it does look like different data sets. For one search it seems to be some seeing the current and some last month's. Last night a site that's been at #5 this month looked like that to site site owner, yet what I saw looked exactly like last month, at #10. To both, even after repeated tries. Today I'm again seeing it at #5. It looks like this is just something we'll have to get used to, it's just hard to explain to people who aren't familiar with it.

[jetson77]

I have a single page that I have made as a place holder for a future site with links etc until my site goes live. I linked to it from a couple of other page rank 6 sites that I have.

Within days literally, it was ranking no.2 for my target keyword. It had a fresh tag beside it. Now just 2-3 days later it has completely gone!

Could someone please inform me on if I have done something wrong, or if this is usual. I am suspecting that it has dropped out because it is no longer fresh, but wasn't in the actual database yet. I am hoping I am not being penalised.

Thanks in advance!

[Beachboy]

Sites that get into the index between major monthly updates aren't always seen in the search results. They tend to fade in and out of the index. Probably it will be permanently added with the coming update. Relax.

[Jane_Doe]

I had a new page that the same thing happened to. In yesterday - made my first sales from that page. :) Today it is out. :(

[Polarisman]

Glad to see that I'm not the only one this has happened to. Even though I had been forewarned that this might happen, it is disconcerting to find one's site gone from the index.

Let's hope that the next dance comes sooner rather than later!

[Jane_Doe]

I had the same thing happen a few months back to a page that I'd spent a lot of time working on and had gotten into DMOZ. At one time it was #5 in the Google index for it keywords, then - poof - it was gone. It came back and has stayed around #3 or #4 since then. The first time it happens is a bit unsettling, especially with all of the talk of PR0 penalties around here. But for new pages it is not an uncommon occurence.

[teeceo]

hi all. If I add some links "today", before the dance, will they show up in "this" update are, the next?

teeceo.

[Marcia]

teeceo, best guess is that if you add them now they'll get crawled this coming time and show up the following time. If it's sooner it'll be a bonus, but stay calm if they don't show up this round.

[Beachboy]

Stay calm? In the optimization biz? Marcia, the SEO community alone comprises 42% of sales of tranquilizers worldwide. Or so I read. ;)

[esset2002]

Hi,

This month something that I never have seen before happened. A new site which I put up in the end of last month (hoping to be in this months update) started geting traffic from google. I was surprised and checked google. And to my surprise the indexpage was in the database...this was one week ago. Today I checked and another 30 pages had made it into Google.

Anybody else experienced this?

ESSET

[Powdork]

yes, but then they went away (or back to their old ranks). This was my biggest jump ever (from 35 to 11 for best phrase). I'm relatively sure we'll be back at 11 in the next few days.
Four months ago we were at 200+. Three months ago we were dropped completely (way upstream provider down during crawl) and that was the best thing that could have happened cuz I learned. Now were at 11 (fingers crossed) and hope to be much higher after the next major crawl.I just have one request for Google Guy. For the next month could you please raise the default setting for results per page to 11? :)

[semick]

I have noticed this phenomenon several times, I add a slew of new pages, [with inbound links from PR 6 & 7] - they show up in google within 1-2 weeks, then they disappear again, until the major update...Is this Everflux?

Scott Emick

[Dave_S]

Hi

I have a new site that google has picked up but its only displaying the main index page in the results, is it normal to have a delay before the rest of my pages are showing up on the results? And if so how long does it normally take for the rest of the site to appear!?

The site first appeared on 10th Sep, then is disappeared for a few days and now its back... but still only the main index page...
(i have text links to other pages from index page!)

Thanks for any info, im new to this and find your postings very valuable!

Cheers
Dave

[Marcia]

Welcome to WebmasterWorld and our Google forum Dave.

As you read back, you'll notice that mid-cyle adds can drift in and out; there's steady fluctuation all the time now. It's hard to know a specific for a site, it would depend on how many pages they got and when they got them. The best indicator is seeing what's happened with other people in similar circumstances. A lot depends on luck and timing.

[lazyz]

It appears one of my domains disappeared from it's number one listing on www but is still in at number one on www2 -

Just the never ending ever-flux or what?

[dazz]

i wouldnt worry...the update is coming in the next few days and the results from www2 usually are on www within 2 days.....if your on www2 google knows your there so just wait a couple of daysand dont panic.