homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

This 94 message thread spans 4 pages: < < 94 ( 1 [2] 3 4 > >     
Everflux - October 2002 mid-month Google spidering and minor updates
Tracking Google activity between regular monthly updates

 7:28 pm on Oct 4, 2002 (gmt 0)

Every month we have a lot of reports and questions about mid-month additions and changes in the Google index. Last month we started consolidating them into one thread so all the information would be easier to find and be especially helpful to those new to Google or our Google News forum.

You can refer to September's Everflux / mid-month changes [webmasterworld.com] to see the previous pattern, which is one of the primary reasons we're trying to keep them all together.

To recap:

1.The Monthly Update
There's one major Google update per month, starting somewhere between the 20th and the 28th. It generally takes several days to settle, at which time the data migrates to Google's partner sites.

2. The Monthly Crawl
The major crawl takes place at right around this time. With some it's been reported to start at the same time or just before the update begins, while others are not crawled until afterward. This continues for several days, and there's been some conjecture that PR might possibly have something to do with the timing.

3. Mid-month activity, referred to as Everflux.
All during the month between updates there has been some spidering, sites added, and minor position changes. This is related to Google keeping their search results "minty fresh" but while minor ranking changes occur, this does not affect the entire index.

It's worked well, so we'll continue to do this now, in October. This is where we can post what we experience and observe, to compare and discuss it among ourselves. We'll again confine it all to one thread so it's simpler to keep track of the information and observations we share that are so important to us all.

Note: We'll be moving some threads onto this one, as we did last month, as well as for the update itself.

We sincerely thank you for your contributions to our collective knowledge and understanding and wish you all good luck with your sites.



 6:10 pm on Oct 5, 2002 (gmt 0)

Right, but the question is, if google only crawls about 2% of my site (which is the current case, though she looks to be going deeper this crawl, so far) how does she know if the page is new or updated? I've I add a new movie (which makes a new page), Google indexes it within 48 hours. If I update a page that google's never crawled, it doesn't actually make a new page, but the page does go onto the list of "new pages" that google crawls to find my new pages. If the page has been updated and isn't new, it's a duck shoot as to whether or not it gets indexed.

So the question is, How does google know what's new and what's just updated? See?



 5:16 am on Oct 7, 2002 (gmt 0)

I just had a site added to the index a day or three ago. I read the faq, but cannot find clarification on a particular question. To summarize, the site rank is fair, but there are quality inbound links that haven't been accounted for yet. Note that the inbound links are how Google discovered the site.

My question is, is it likely to assume that once my PR is recognized that the ranking may improve slightly, or is the current ranking which indicates a gray PR0 (not yet indexed) fairly accurate?


 10:29 pm on Oct 13, 2002 (gmt 0)

I just got a hit from a keyword search on google, when I linked from the logs my page was gone then I made www3 search and I was on the next page, maybe we have a little everflux here.



 10:49 pm on Oct 13, 2002 (gmt 0)

FWIW, Google stopped by my new site on Oct 2. Hit it harder on the 3d and really hit it on the 4th. The home page showed up in Google serps today. Still showing a gray bar (No PR).



 1:27 am on Oct 14, 2002 (gmt 0)

kgormat, what happens in between isn't necessarily an indication other than that Google found you. You won't be able to tell until the update, when the links and corresponding Page Rank come into play. That will be your starting point, not until then.


 4:17 am on Oct 14, 2002 (gmt 0)

Grumpus: perhaps googlebot remembers url's that it has decided
not to crawl (i.e. already on it's list). When you add a new
page, the url is 'new' to google (e.g. ID number has never been
seen), so googlebot thinks it is super fresh, and puts it at
the front of the list to crawl.

To rephrase, google guesses that a page is 'updated' because the
url has been seen by googlebot before, and 'new' when it is
an unseen url.

I, too, have been unable to figure out a pattern to google's crawling. I have 200,000+ pages, with only a portion crawled.
Some pages have been crawled in one month, only to fall out
of the index a few months later, even though the site navigation
hasn't changed.

>>>So the question is, How does google know what's new and what's just updated? See?


 1:55 am on Oct 15, 2002 (gmt 0)

From looking at my logs for several sites (PR 5,6 & 7) freshbot has a very obvious pattern:

1: Grab main index page
2: Follow all internal links listed on the index page
3: Follow any new internal pages it's not seen before

I've yet to see freshbot go deeper than that internaly although it's possible it follows any new external links it's not seen before regardless of how deep that page is in relation to that sites index page. Any thoughts?


 4:40 am on Oct 15, 2002 (gmt 0)

I've got an October 12th date next to my Google listing with the host's admin page in the cache, and my title and description as:
venus.powweb.com- venus.powweb.com
which is a bit of a disaster if it sticks.

What I want to know is: might googlebot revisit my site between now and the next update? And is there any help I can get from Google? (or is prayer my only recourse?)



 2:04 pm on Oct 17, 2002 (gmt 0)

There are big chnages in the ranking since I last visited Google yesterday evening. Nearly after every URL is the date (16. oct. ) shown.
Dou you think this is an update?

creative craig

 2:09 pm on Oct 17, 2002 (gmt 0)

I am seeing differences to, but not enough for a full blown update I think!

May just be the everflux acting up a little more than usual.



 2:24 pm on Oct 17, 2002 (gmt 0)

Here, some SEO agencies have officially declared it was the update.


 2:27 pm on Oct 17, 2002 (gmt 0)

No update yet Webbie.

creative craig

 2:29 pm on Oct 17, 2002 (gmt 0)

If thats the update it was the smallest and the quickest ever!


 6:42 pm on Oct 13, 2002 (gmt 0)

a couple of my sites just lost their ranking in google today. they're still in google, but don't show up for any results.. plus the cached versions have been reverted back to the state from like last month I think..

I'm also out of yahoo now, too.

what is going on?


 7:06 pm on Oct 13, 2002 (gmt 0)

What you are probably seeing is called "everflux". A good description of it can be found here [webmasterworld.com].


 12:22 pm on Oct 14, 2002 (gmt 0)

ok, but why is my sites description reverted back to the state from a month ago or so. is it going to go back to the current state or is google travelling back in time?

are they providing results for archive.org now?


 1:26 pm on Oct 14, 2002 (gmt 0)

I've seen this happen to new sites.


 1:33 pm on Oct 14, 2002 (gmt 0)

Seeing it all the time a site of mine last month was ranking well, after the update vanished, now it's back again although I'm glad to see it this is very confusing


 1:43 pm on Oct 14, 2002 (gmt 0)

thanks for your replies. from your experience - do the sites return to their correct (up to date) cached state and the previous ranking or rather not?

is it an error in the Google database?

what are the experiences of other members?



 1:10 am on Oct 15, 2002 (gmt 0)

Hmm... if google captures new data that is more relevant to the keyword search you inputed, then it's logical to say the new sites will go first. So, I guess Google isn't the one to blame. The new page that Google captured is... sorta...

I don't target at just a few keywords. When I work my site, I target over thousand. So, I don't have to worry about a few changes in Google ranking change since each page accounts for a small amount of traffic.


 2:51 am on Oct 15, 2002 (gmt 0)

As Irock mentions "targeting key words"... I've found with the new update that being number one for one key word isn't worth as much as being in the top 20 for many different key words or terms. I have a couple domains that pop in and pop out of the top twenty. I've also seen more traffic from Google than ever before. My sites are coming up for more search terms more often than any other update. Granted, my sites aren't number one anymore (they use to be for only one or two key words) but I'm getting more traffic than any other update. Sometimes just because you lose ranking doesn't mean you will lose traffic. I was really shocked when I saw my number one rankings disappear but after a few days I noticed more traffic...


 10:12 am on Oct 15, 2002 (gmt 0)

I know what you mean, guys but my sites were exactly like that. They weren't targetting one keyword. I was getting traffic for many different keywords, and I also think this is the best way to go.

HOwever, the problem is, my sites are nowhere to be found now.. One was an old site, and the others were new. The old one shows up in google the way it was about a month ago (title, description, cache). Kind of like www.archive.org

I think it's an error in their database, I just hope it doesn't stay this way. I mean why would google go back to the previously cached version of the page. This means displaying OUTDATED RESULTS.

Any thoughts?


 7:11 pm on Oct 15, 2002 (gmt 0)

I have noticed that my new website has just like yours dissapeared from main index. It was ranked on multiple keywords 50+ with rankings raging from 1-10 position. It was in the results for about 3 weeks now it has completely wanished. The site even when it was ranked had gray PR bar in toolbar, and it is still in www2 google index, so keep my fingers crossed that this is just "everflux" and that my site will finally get some PR (should be at least 5) with the next update and the rankings come back...

johnd, did you check if your site is still in www2?


 7:18 pm on Oct 15, 2002 (gmt 0)

They did it to my site too. Google deleted it after 15 months on 28th August and now Yahoo have deleted it too. I have emailed both of them and neither have replied why.


 8:09 pm on Oct 15, 2002 (gmt 0)

johnd, did you check if your site is still in www2?

my site is in all indexes, but it shows up as a cached version from a month ago (I had changed the title and the content slightly in the meantime). now, when I type the domain in the google search bar, I see the title that I used to see in google about a month ago. It had been updated by google, but now it's back to the previous title etc.

plus, my site is burried somewhere so deep that I can't even get to it for the keywords that I used to be in top 10-20 before.

but the answer seems to be simple - it's an error, like some others I've seen. showing old cached version of the page after having shown the new one for like a month can only be an error


 11:21 pm on Oct 15, 2002 (gmt 0)

What I'm seeing today is Everflux Plus. After the last update there was a pretty big change with one of the searches I watch. It stayed well into settling of the update, then there was a shift a few days later. Let's call them Version 1 and Version 2.

I checked that search earlier today, and the first time I saw Version 1, exactly as it was when the update first "seemed" to settle. I checked again and got a Version 3, with one site dropped down a bit out of #1 and a few other pages gone.

I'm now getting Version 1 with the toolbar, with 4 of my pages in there and 596K pages, looking at www2 with 546K pages with 2 of my pages still there and 2 gone, and at www with 616K pages all 4 are there.

With another search, all 3 yield the same, but for one of my sites the indented page is now a different one.

Incidentally, mine are not all the same site, and there don't seem to be any fresh dates right now. Version 3 can't be pulled up, but it's a predicted wild guess that it'll turn up on www later tonight with the dates showing, with one page dropping down and a couple of others out of that search.

These are all sites that have been in the index for a long time, they're not new and none has ever had a penalty. So the appearing/disappearing phenomenon doesn't seem to be restricted to only new pages or sites. It also doesn't appear that disappearing pages is an indication of a sudden unexplained penalty or serious error, though disappearing pages is a little fresher than some of us enjoy.


 12:43 am on Oct 16, 2002 (gmt 0)

These are all sites that have been in the index for a long time, they're not new and none has ever had a penalty.

Marcia, maybe this is a reason for the pages to be disappearing..

Think of the 'mintyfreshness' concept, and enlarge this to all the possible consequences.

Maybe this is not your case, I talk from my own experience :)



 3:31 am on Oct 16, 2002 (gmt 0)

Hello Guys,

I submited my site to google and on Saturday the 11th of October my site started coming up under certain keywords, it was actually #4 for a certain keyword. The same thing happened on yahoo, it was #4. But since saturday night my site has not been on google OR yahoo...it is a new site so I am not sure what happened...

If my site is blacklisted how would i know?

I find it really odd that my site was up one night and never again since...it is a bit odd to say the least.

If anyone could help me out I greatly appreciate it!



Visit Thailand

 4:14 am on Oct 16, 2002 (gmt 0)

After the last update one of pages went from 3 to 1 (first page) the page is very old and has been around there for the last few years. Then yesterday this same page has dropped to 6 on the first page but in www3 it is still number 1. I have seen this happen in the past and then after a few days it reverts back to number 1 or whereever it was before. BTW in case you were wondering the page is internal and pr7.


 4:37 am on Oct 16, 2002 (gmt 0)


Patience. Sites submitted since the last update (Sept. 28th I think it was) are only temporarily in the index. They will jump in then jump out, then back in. It is erratic as all get out, but don't get nervous. That's just the way it is. Your site should be much more firmly in the Google index with the coming Oct. update. Relax. Go to a movie.


 5:10 am on Oct 16, 2002 (gmt 0)

Psst... Skirope:
I've seen this happen to new sites.

This 94 message thread spans 4 pages: < < 94 ( 1 [2] 3 4 > >
Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved