Welcome to WebmasterWorld Guest from 3.80.4.76

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Google isn't indexing new content

     
8:52 am on Apr 18, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 20, 2004
posts:98
votes: 0


Every day I add new content to my website. It used to got indexed within one week or so. Now it is almost two months ago since new content was indexed for the last time.

I checked my robots.txt for failures and my htaccess but nothing strange. My old content is also still indexed. What can be the cause?

11:53 am on Apr 19, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 20, 2004
posts:98
votes: 0


Amen
12:01 pm on Apr 19, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member tigger is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 16, 2000
posts:3560
votes: 0


that does seem to sum it up Rich
12:04 pm on Apr 19, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:689
votes: 0


And as always, no word from the Plex?
12:08 pm on Apr 19, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member tigger is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 16, 2000
posts:3560
votes: 0


something that I can't get my head around is how a page can have PR(4) but not be cached! anyone got any clues?
1:26 pm on Apr 19, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:689
votes: 0


And Dayo_UK, I can see the problem on a site that never faced a supplemental or canonical issue..
1:39 pm on Apr 19, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 6, 2004
posts:84
votes: 0


Add a new page link to your index page and Google will now take longer to cash it. Any links to pages off it and its a few weeks minimum.

What concerns me, though, is that there are some sites that do get their pages indexed rather quickly. That is what makes me wonder if this is a leftover issue from the supp hell bug many of us went through.

2:20 pm on Apr 19, 2006 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Feb 25, 2004
posts:1003
votes: 47


Within the last month I posted a single new page. I linked to it from my home page and it was in the SERP's within a few days (rank #31).

During the following days I started creating internal links to it from other pages on my site. Half way through this process the page climbed to #26.

By the time I had finished linking to this page internally (the whole process probably took over a week) the new page then fell totally out of the top 100 in the Google SERPs. Who knows?

Dayo_UK

2:35 pm on Apr 19, 2006 (gmt 0)

Inactive Member
Account Expired

 
 


wanderingmind

If you are able to sticky me the url of that site I would be intrested - no worries if you dont want too.

Personally, I *still* think that it is due to the cross-over from the normal Googlebot ranking to the Mozilla Googlebot ranking.

If crawl depth, rank and indexing is supposed to be based on PR then there is little doubt in my mind that the PR generated by Mozilla Googlebot so far has not been applied. (Even when it was applied to the TB is was a bodge - whether it has been applied to the ranking, crawling and indexing of sites at all is what I am not sure about)

2:59 pm on Apr 19, 2006 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts:12172
votes: 61


By the time I had finished linking to this page internally (the whole process probably took over a week) the new page then fell totally out of the top 100 in the Google SERPs. Who knows?

No one does really. And, given the time factors being quoted within this topic, I don't think enough time has passed to really make any assumptions. One month, two months, etc. If the sites in question are fairly new (within the past 6 months), it's going to take a bit before things settle down and you start to see consistency in the results of your changes.

And again, no one really knows. Each and every one of us has a different circumstance and there is no "one answer" for all. Some may be dealing with whatever the Sandbox is. Others may be dealing with technical issues. And others may be dealing with penalties. Who knows? ;)

Personally, I find that watching this stuff is like watching paint dry. One page? Come on now. What about all of your other pages? The Infoseek days are gone. :)

3:04 pm on Apr 19, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member tigger is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 16, 2000
posts:3560
votes: 0


if it was just one page then I'd be happy but for me its anything new just not getting crawled despite being very well interlinked into the site
3:05 pm on Apr 19, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 23, 2006
posts:88
votes: 0


OK, here's my latest wacky theory:

One of the mega-brains at Google recently introduced a mega-bug into the PR scoring mechanism that resulted in millions of new pages being granted an artificially high PR. Hence, lots of accounts of brand new pages with innexplicably high PR values (often higher than the page that linked to it).

The end result is a bunch of confused and overstretched Googlebots that are unable to schedule a decent deep crawl of many sites due to the deluge of artificially high priority new pages.

It would be nice if Google could go a week, or even two, without introducing fresh bugs into the dreadful mix that is Big Daddy.

All conjecture of course...

3:23 pm on Apr 19, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:689
votes: 0


Dayo, done.

Dayo_UK

3:28 pm on Apr 19, 2006 (gmt 0)

Inactive Member
Account Expired

 
 


Yep, thanks replied by Sticky.

Just so everyone else knows - the site definetly has Canonical issues - still have not seen a site with this problem (or the supplemental) that does not have Canonical issues.

Same problem that has been blighting the index for so long.

3:33 pm on Apr 19, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 23, 2006
posts:88
votes: 0


Just so everyone else knows - the site definetly has Canonical issues - still have not seen a site with this problem (or the supplemental) that does not have Canonical issues.

Sorry to be a pain. But could you please give a very precise definition of exactly what you mean, i.e. the definitive test that you just did to arrive at the "yes, this site has canonical issues" conclusion.

The reason I ask, is that by my yardstick, my site does not suffer from any canonical issues, but has suffered greatly at the hands of Big Daddy - 1000s of dropped pages (but no supplemental issues).

Dayo_UK

3:37 pm on Apr 19, 2006 (gmt 0)

Inactive Member
Account Expired

 
 


Clint

There are different ways of testing the site - some can be a bit obscure. As it is slightly harder with the latest PR update to see.

For the site that wanderingmind stickied me - it was easy as both the non-www and the www homepage were cached and indexed by Google.

4:35 am on Apr 20, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:689
votes: 0


That's right ClintFC. Both were indexed by Google, and for some time too. (Just that so far there were no ranking issues despite it). But if one just looks at Dayo's theory that its a site with a canonical issue that is facing the latest no crawling problem, yes, my site fits in.
8:45 am on Apr 20, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:Oct 1, 2004
posts:607
votes: 0


On several "quality" sites I monitor, it seems to be taking longer for pages to be indexed (typically 2~4 days rather than the accustomed 1~2) and even longer for them to be cached (about a week).

I'd guess (speculate) it's teething troubles stemming from the new bot infrastructure or whatever they've been twiddling with.

11:20 am on Apr 21, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:689
votes: 0


What if Google is just trying to reduce their bandwidth usage? :-)
1:17 pm on Apr 21, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 31, 2004
posts:786
votes: 0


How about this

NOTHING is wrong with google or its algo

Do webmasters here not think that all this is pure deliberate action on behalf of google to try and push up adwords revenue?.

Its that simple imo

Look at the recent results. Last quarter revenue was up by 66% over expectations. During this quarter they have NOT indexed all sites fully, not listed webmasters pages, had poor serps, dropped pages from authority sites, not indexing new content etc etc etc.

The net result of all this has been webmasters moaning in here about it (me included) and a sharp increase in revenue for Google. Whos the idiot?

The only problem with the serps being so bad due to Google doing taking this action is that they risk losing market share SHOULD msn or Yahoo get it right, however i think i have more chance of having tea with the Queen Mum than one of Googles competitors taking any market share off them.

Google dominates, webmasters have put them their and we are reaping what we sow as a result

2:25 pm on Apr 21, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:689
votes: 0


RichTC,

That would work only for pure commercial sites that need to sell. A drop in rankings wuld mean they are tempted to use Adwords.

However, I suspect that the larger number of sites would be informational - huge news websites for example. There is no way they can use AdWords as they are not selling anything.

2:33 pm on Apr 21, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 6, 2004
posts:84
votes: 0


Personally I think this sounds more like a conspiracy theory than anything else.

If it was the case, this problem would be ocurring across the board instead of just select sites.

4:41 pm on Apr 21, 2006 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:June 4, 2002
posts:1911
votes: 3


For those who aren't getting new pages indexed -- do you use straight Javascript on your links/menus? If so that may be the reason. I use a combination Javascript/html/css menu and my new pages get indexed within about 2-3 days and gain PR quickly too.
6:17 pm on Apr 21, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:689
votes: 0


Nope. No javascript on sites I know are facing the problmem - straight HTML links.

Also, just noticed that 3 more sites I know (not mine) have lost pages in Google (and one of them lost some pages, rest turned supplemental).

Very cheering.

This 53 message thread spans 2 pages: 53
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members