Is there an update going on?

[willybfriendly]

I see some pretty significant changes in the SERPs across a number of terms. Also note that the cache is showing pages I updated within the past week.

I have noted more than the usual shifting of pages in and out of the SERPs for the past week or so, and of course we have seen threads about updated backlinks (FWIW).

Anyone else seeing changes?

WBF

[Powdork]

Wow,
One hour before MSN debuts their new search technology.
Coincidence?

Still no appreciable changes to the serps in my area. Seems like there's 4 billion urls sitting idly on a hard drive somewhere. I am looking forward to this weekend however. I want to like Google again.

[promis]

"The Google home page announces 8 billion pages indexed."

I believe it is a deception. My site is shown to have 5000 indexed. It has - and ever had - no more than 2200.

[edited by: promis at 4:17 am (utc) on Nov. 11, 2004]

[jnmconsulting]

It just so happened that I have been watching that number on the main page. I has been stagnant for over 3-4 months maybe more. I have been checking as well using +the and have seen the changes and the big jump started yesterday, maybe the day before. My guess is that the number on the front page was left there at the 4.2 billion or so to throw everyone off. I'm in hte USA central time zone, the change took place I think around 5:00pm this evening. It appears that BLOG was posted at about that time.

[shri]

Ok. Here's why there are no changes (my simple minded conclusion) in the serps.

The rolling PR calculation algorithms have not caught up with the index. Over the next several days / weeks, the PR gains, anchor text and other off page factors from the new index will be factored in and will contribute to changes in the results.

What am I on? A Starbucks crappachino.

[Vimes]

Hi,

8 bill it is yahoo's pages have doubled so have the major News sites.

strangely yahoo pages are all url only for me.

Vimes

[ILLstyle]

me too

[steveb]

C'mon, it means nothing. All my sites show some incrase in pages... but for the smaller ones where all of the pages are always lsited, the increase is 100% due to URL only listings like site.com/directory that duplicate a site.com/directory/ page that is also listed (note the forward slash), or a significant increase in supplemental pages that have long since been deleted.

As of this moment absolutely nothing has changed, except Google is showing signs of more broken-ness.

[Kirby]

>C'mon, it means nothing.

Sure it does. It means Google continues to be the repository of psuedo-directory crap put up for the purpose of laundering PR.

It takes waders to slosh thru the sewage that passes for results and ranks only because of keyword slammed page titles and anchor text from interlinked pages on the same site.

[Vimes]

Steveb,

except Google is showing signs of more broken-ness'

yea i reckon theres a panic going on with Msn releasing the beta sooner than thought.

kirby,

It takes waders to slosh thru the sewage that passes for results and ranks only because of keyword slammed page titles and anchor text from interlinked pages on the same site.

couldn't agree more MSN are more relivent for the search terms that i review.;)

Vimes

[edited by: Vimes at 6:37 am (utc) on Nov. 11, 2004]

[BReflection]

searching google for +"the" returns 8,000,000,000 pages. 3 days ago it was 6,100,000,000.

[Powdork]

As of this moment absolutely nothing has changed, except Google is showing signs of more broken-ness.

I'm guessing this will be the first of a three part process.
1. Start with an all encompassing base index.
2. Blend in new data which will remove nonexistant pages and associate data with existing url-only-indexed pages.
3. Apply algo changes to finalized index.

If so, it will certainly steal some thunder from the MSN debut (at least in this neck of the woods).

[Rugles]

The result for one of my most competitive keywords was 6.5 mil, as of yesterday it is 11 mil.
Luckily, I have maintained my great position in the SERP despite losing a tonne of backlinks. So the trend continues that the backlinks we are shown are next to useless.

[TinkyWinky]

steal some thunder from the MSN

Have to be a damn site better than it has been the past few months to do that. All they are doing is updating (albeit on a big style) whereas MSN have 'new shoes'!

[BillyS]

Come on guys admit it. Google is a leader and they are not going to sit back and wait for MSN to catch up.
If I look at this objectively, Google is a leader and just made a big move to stay that way. MSN is trying to play catch up but the leader just pulled away again.

One thing is for sure, the quality of search results will improve and to me that is good news.

And for the record I like Microsoft and Google. The race will benefit everyone that is looking for higher quality search results.

[g1smd]

A search I check from time to time reported 10 million results a year ago, and grew to 12 million results many months ago.

Less than a week ago it suddenly reported 18 million results, and tonight it is up to 22 million results.

There are a LOT of 9 Nov and 10 Nov 2004 Fresh tags appearing in the last few hours (even for pages that have not had fresh tags for months).

For pages that have not been re-cached for a long time, a LOT of them are showing a cache date of 31 Dec 1969 23:59:59 GMT (which is the Unix Epoch minus 1 second).

[wiskid]

I have noticed that Google and AOL often have very similar results, but it varies form search term to search term, at least between the ones I follow. With this up date I observed some strange things where AOL and Google suddenly showed very different results. GG improved drastically while the same search on AOL dropped several sites way down.

Has anyone else seen this or have any clues as to why they are suddenly so different?

[g1smd]

The 22 million results yesterday, is up to 23 million today (see 3 posts back).

[The Contractor]

For those of you seeing more pages of your site than there should be... any chance they have now included pages in the index they should not have?

I have several thousand extra pages (6,340 to be exact) in this new larger index. Problem is they are CGI files that have always been blocked from day one via robots.txt. The whole cgi-bin has always been blocked (3 years) and up until now Googlebot adhered to the robots.txt file (which is a valid robots.txt). It sucks that they have gone where they shouldn't have to dig up completely useless pages....

Now I guess the only way to get them out is to contact them... but with my luck they will throw out the whole domain in their haste...hehe I guess I will leave it alone since it looks worse on them than it does me..

edited: for clarity

[edited by: The_Contractor at 4:32 pm (utc) on Nov. 12, 2004]

[The Contractor]

Been looking around doing allinurl searches on some large companies (IBM, Adobe, etc)and looking at their robots.txt file .... I see Google has not adhered to theirs either. Even WebmasterWorld has a few pages in that Google shouldn't have in...

[mark1615]

Not adhering to the robots.txt file? And this is the company that constantly takes holier than thou positions about SEO? I for one could do without the posturing.

[webhound]

The only thing Google cares about these days is AD revenue. They sure don't care about the quality of the SERPs. Get real, with all the PHDs working there you can't tell me that this is the best they can do given the sites that continue to rank well month after month. Blog backlinks, forum backlinks, and other SEO techniques continue to manipulate the STALE Google index. I don't care how many billions of sites they have in the index, if it ain't organized WHO CARES?

[jimh009]

> For those of you seeing more pages of your site than there should be... any chance they have now included pages in the index they should not have?

Not on my site, at least. Google has unlimited access to the site - and it is a static site. I've seen my site doubled in size in the big G - to over 8000 pages (the site itself is only 4000 pages).

[mark_roach]

For those of you seeing more pages of your site than there should be... any chance they have now included pages in the index they should not have?
I have several thousand extra pages (6,340 to be exact) in this new larger index. Problem is they are CGI files that have always been blocked from day one via robots.txt. The whole cgi-bin has always been blocked (3 years) and up until now Googlebot adhered to the robots.txt file (which is a valid robots.txt).

I am seeing similar results for my sites and I have cgi-bin blocked.

However I don't think that the content of the page is actually indexed. None of my cgi-bin pages are cached and when I search for text that is on these pages they are not returned. I think that google has merely logged the existence of the page in the database. Hence the increase in numbers.

[esllou]

same here, 200 cgi-bin pages even though cgi-bin has been blocked via robots.txt for THREE YEARS and has never caused a problem before.

and I'm not happy either!

[Spine]

In their haste to include 'everything AND the kitchen sink' I'm seeing that some mid-nineties style doorway pages are doing well.

Top drawer all the way G.

[The Contractor]

I am seeing similar results for my sites and I have cgi-bin blocked.
However I don't think that the content of the page is actually indexed. None of my cgi-bin pages are cached and when I search for text that is on these pages they are not returned. I think that google has merely logged the existence of the page in the database

Yeah, except the whole idea of blocking bots are so they don't go anywhere they are not supposed to. Like to hear from Google on this one since I really don't need them keeping a publicly viewable inventory of my perl files, and I'm sure others feel the same way.

[Dayo_UK]

For those of you seeing more pages of your site than there should be... any chance they have now included pages in the index they should not have?

Yep hundreds. All blocked by robots.txt

Only listed as url entries - as google obviously knows they exist but can not crawl the page. So I guess Gbot is obeying robots.txt.

But not sure why Google think listing url only pages adds value to their database.

[webhound]

well like i've said before, what we are seeing in G these days just doesn't make sense. it's like google is broke.

[The Contractor]

So I guess Gbot is obeying robots.txt.

How do you figure that? They are not supposed to even go to that folder/directory if it is blocked...

[steveb]

"They are not supposed to even go to that folder/directory if it is blocked"

They don't "go" there. They just list links they see on pages they are allowed to go to.