homepage Welcome to WebmasterWorld Guest from 54.227.67.210
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Visit PubCon.com
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

This 38 message thread spans 2 pages: 38 ( [1] 2 > >     
No Indexing for the Last 2 Weeks On 100+ Sites
Google brandy
webnewton




msg:130092
 1:08 pm on Feb 25, 2004 (gmt 0)

I manage a group of 100+ sites. None of them has seen the spider for atleast 2 weeks now.This is somethig strange as i've noticed them being regularly indexed for the last 9- 10 months. Few clues;

1)I'd done a I.P reshuffle for all my sites around a month back. But i dont' think this created some probelem as bot kept coming at the sites even after. Bot also registered the changes in the cache corrosponding to the changes done on the sites.

2)Barring a few sites almost the whole group was penalised during florida.

3)I've also noticed a considerable cut in the backword link from google last week. I know that this is nothing new. Google often cuts back the backword links. What was strange however was the cut back in the no of links shown with the command "+www.site.+com"

4)Also though most of my site have returned to the top position in google, the titles reflected by google are preflorida. Strangly google had registered the post florida title changes for these sites.

Any comments?

 

Grace




msg:130093
 12:42 pm on Feb 27, 2004 (gmt 0)

Hi webnewton,

Even i am experiencing similar kind of situation.The Title and Description displayed are Pre Florida.We were in top positions after Florida and Austin but after Brandy there is a huge drop in the positions.

mr_strong




msg:130094
 1:58 pm on Feb 27, 2004 (gmt 0)

My sites (3 of them) are experiencing similar things. Google has just grabbed the index page in last 2 weeks but gone no further. 1 of these sites is new, the other 2 are well established.

jbgilbert




msg:130095
 2:07 pm on Feb 27, 2004 (gmt 0)

Similar issue here...

Googlebot crawling on "some" sites is not as aggressive as it used to be,

Some backlinks have not been updated since early January -- especially those coming from internal site pages,

PR updates on internal site pages have not been done since early January.

Question is: Is it by Google design or does Google have a new problem?

SEOPTI




msg:130096
 5:29 pm on Feb 27, 2004 (gmt 0)

The same issue here. Seems like a big Google spider slowdown. Maybe they are having spidering problems.

Seattle_SEM




msg:130097
 5:59 pm on Feb 27, 2004 (gmt 0)

I have seen a ton of spidering, last weekend, but those pages spidered have not been updated....

steve40




msg:130098
 6:08 pm on Feb 27, 2004 (gmt 0)

Also noticed significant slowdown in spidering across 6 sites with new pages added , I don't add hundreds or thousands of pages a week maybe 10 - 20 maximum each week per site but as i have a google site search box for my visitors to search mysite am starting to feel concerned if I should go back to picosearch
any ideas please GG if this will be standard crawling from now on , no problem if it is but would help to know

steve

wellzy




msg:130099
 6:24 pm on Feb 27, 2004 (gmt 0)

I've been spidered plenty, but have seen no update. My new pages have not been included and PR has not been given to the new pages. They were made about 2 weeks ago. I am however being spidered deeply.

metrostang




msg:130100
 6:48 pm on Feb 27, 2004 (gmt 0)

Googlebot was spidering my site for the first two weeks of February, must have indexed all 17,000 pages. The index is the only page so far to show a fresh date.

If GoogleGuy is listening, when will the database be updated? Many of us have made changes and want to see the results. I used one area of my site to experiment with stemming and will apply to the remainder if sucessfull.

I've read others say new sites are showing up, but have not seen any new pages other than index since first of February.

phantombookman




msg:130101
 7:07 pm on Feb 27, 2004 (gmt 0)

Hi
I am getting new title changes showing in 2 days! Being crawled more often that ever before but have not had any new pages added to the index for about 6 weeks.

My keyword searches had gone haywire as well some get many get #1 hits others show nowhere at all?
Regards
Rod

sdani




msg:130102
 8:52 pm on Feb 27, 2004 (gmt 0)

May be the 4 billion mark is the answer to these questions. Googlebot is busy crawling new sites and increasing the number of pages instead of crawling the existing pages more often.

farside847




msg:130103
 9:43 pm on Feb 27, 2004 (gmt 0)

I am seeing smaller than normal googlebot activity too. About 10% of normal - oddly enough, looks like only PR5 or above pages being hit....

caveman




msg:130104
 10:41 pm on Feb 27, 2004 (gmt 0)

Same observations with some of our sites...

looks like only PR5 or above pages being hit

...that too...I see only the homepage showing fresh date. :-(

darkroom




msg:130105
 11:36 pm on Feb 27, 2004 (gmt 0)

i am observing something similar...I recently redesinged the frontpage of one of my established sites, and now its showing a month old cache... freshbot (66....something)has stopped visiting the site..however the deep googlebot(216...something) is still visiting....

SEOPTI




msg:130106
 12:28 am on Feb 28, 2004 (gmt 0)

Maybe they just don't have enough servers and capacity to spider all sites?

Stefan




msg:130107
 12:36 am on Feb 28, 2004 (gmt 0)

64.68.xx.xx is the bot. 216 is now just translation isn't it?

Our site has many of its pages at PR5. They are getting crawled harder than ever. The rest are mostly PR4 and are getting looked at less often, but still being visited.

I've read a lot of posts lately about people not being crawled. Believe me, it's not a universal problem. We're getting crawled and fresh-tagged like never before.

darkroom




msg:130108
 4:17 am on Feb 28, 2004 (gmt 0)

my site that is showing 1 month old cache now...it was getting crawled a lot harder(was showing new freshtags on about 300 pages everytime) before i redesigned the frontpage...its only after i uploaded the new design that my site isn't getting crawled anymore....its really wierd...i dont know what to do now..

Stefan




msg:130109
 4:25 am on Feb 28, 2004 (gmt 0)

darkroom, if all else fails, you could always put the old frontpage back and see if that helps...

[edited by: Stefan at 4:38 am (utc) on Feb. 28, 2004]

darkroom




msg:130110
 4:33 am on Feb 28, 2004 (gmt 0)

Stefan,
thats the only option i can see myself with..unless googleguy adresses the problem and explain how to fix it....its been a week now and no sign of any 64.
but interestingly, when i uploaded my new redesigned frontpage, i did get freshtags 2 to 3 times consecutively..but i kept on tweaking my frontpage for getting my same rankings back...instead, not only did i loose the rankings, but also lost freshbot activity...

zgb999




msg:130111
 2:55 pm on Feb 29, 2004 (gmt 0)

An obvious reason is that Google has 1 billion pages more to crawl. This seems to have slowed down the crawling process.

Some pages still are visited daily but from what I can see the content Google is showing is not as fresh as it used to be (on a whole) and new pages take longer to be in the index.

The main question is whether this is a temporary state (until there are more ressources for crawling) or whether this will stay.

With the competition from yahoo! I am sure Google has all interest in speeding up the crawling process again to get in fresher content faster.

steveb




msg:130112
 10:21 pm on Feb 29, 2004 (gmt 0)

Individuals need to understand that they are not the whole universe. Google has dramatically upped both its depth of crawl and frequency of fresh dates. Half my pages get a new fresh date every single day.

webnewton




msg:130113
 11:57 am on Mar 1, 2004 (gmt 0)

With the replies from all of you one thing is clear now. The I.P changes i did on my sites has nothing to do with the bot not updating my sites. Also;

1)Since the last 10 months or so i've seen google updateing my sites every 3 days. If i needed to change any title for some specific occasion i did the changes 3-4 days in advance. I'd changed the titles of some of my sites on 10 feb to introduce a Valentine's special offer. Unfortunatley iven't seen any updations yet.

2)All the other sites who're ranked in the SERPS along with my sites are being updated by the bot regularly.

3)My stat shows me that bot is regularly indexing my sites, indeed a deep indexing as ever before but no updations. The date script that shows below the listing of the site is missing for more than 20 days now.

Should i wait, or there is something i can do about it. Any comments from googleguy?

hawk




msg:130114
 12:37 pm on Mar 1, 2004 (gmt 0)

There is no hard and fast rule. We all know that. Sometimes nothing makes sense at all. My observations:

1) This has not got anything to do with any Google Algo change.
2) The quality/PR/pages etc. of the site crawled does not matter.
3) This is a global phenomenon. Many people are experiencing this.

What could be the cause:

1) Google Servers have problems again.
2) They have not been able to index/cache sites in thoss particular servers/databases.
3) Even the Google Guy is confused/amused.

So the best option is to: WAIT AND WATCH AND PRAY HARD TO THE "GOOGLE LORD OF WEB"

Cheers

michael4




msg:130115
 12:46 pm on Mar 1, 2004 (gmt 0)

Same observations here:

My PR 0,1,2,3 sites are not being updated and a PR 5 site is updated every day.

Since there is no PR updated happening, I am concerned about the new sites as
Low PR - slow indexing and slow indexing - low PR :)

michael

jchance




msg:130116
 12:50 pm on Mar 1, 2004 (gmt 0)

An obvious reason is that Google has 1 billion pages more to crawl.

Google did not overnight add a billion pages. They had been adding those additional pages for months and months and all they did overnight was update the web page count at the bottom of their home page and send out a press release to try to steal yahoo's thunder.

Grace




msg:130117
 2:40 pm on Mar 1, 2004 (gmt 0)

I am facing one more weird problem.The changes we made on some of our Pages got crawled before Brandy and they were reflected.After Brandy i see that google serp's show old pages(pre austin & florida results) and the recent changes made are not at all considered.Is this a problem with google Database.How many days more will it take to reflect the changes? Most funny thing is even for our home page with page rank 6, its showing the pre Austin & pre Florida Tilte.The links that are on Home page are not crawled since Brandy and No changes are reflected.Is any one else facing similar problem?Do you have any solution to make google crawl the changes on the site.

zgb999




msg:130118
 6:06 pm on Mar 1, 2004 (gmt 0)

"Google did not overnight add a billion pages."

You are certainly right but Google has to actively crawl 1 billion pages more now to keep them fresh...

k18boy




msg:130119
 6:25 pm on Mar 1, 2004 (gmt 0)

darkroom! my site is showing really old page in the cache too! so old in fact, it updated the cache to my new page but has gone back to the old one again?

this has ment losin position 1 to now bein at 10!

anyone else have this prob?

is google trin too had to index new sites without keepin up-and-runnin sites updated?

mayor




msg:130120
 6:26 pm on Mar 1, 2004 (gmt 0)

My log analysis indicates the pages getting left out in the cold are the brand new pages.

So maybe the problem is an overload on resources. Once a new page is thrown in the serp database, it will begin requesting update resources on a frequent basis.

Anyone got a few thousand servers they can donate to the cause ;)

ClickMagazine




msg:130121
 7:26 pm on Mar 1, 2004 (gmt 0)

I put in a couple of new sites just before Brandy and during that weekend.

They've been bouncing all around, new Title tag, old Title tag, in, out, back and forth. Just when I think it's making forward progress, now I'm back to the beginning. Old Title tags and the newest of the sites are not in the index. The newest being the ones added after the update began.

Don't understand it and I'm tired of being patient but it does seem like a few of us are seeing the same thing.

Maybe Google needs some Pepsid to settle it's stomach.

This 38 message thread spans 2 pages: 38 ( [1] 2 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved