Welcome to WebmasterWorld Guest from 18.206.168.65

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

New pages indexed, de-indexed, re-indexed - is this normal?

     
8:37 pm on Jun 7, 2008 (gmt 0)

Junior Member

10+ Year Member

joined:May 30, 2008
posts: 47
votes: 0


Most new pages seem to get indexed within 24 hours, then around 40-50% of them get de-indexed again for about 2-14 days then get re-indexed again.

After this process the page is then very stable and is very unlikely to get de-indexed although may drop in rankings as they get older.

Is this normal behavour for all sites as it seems to happen all the time on our site.

The site is 2 years old has plenty of backlinks, hosted in the uk and is a uk site.

Lee

7:02 pm on June 8, 2008 (gmt 0)

New User

10+ Year Member

joined:June 8, 2008
posts:33
votes: 0


Just to let you folks know..

I posted uploaded another page... my lost page was re-indexed and ranked as before and the new page is ranking well..

Regards David

7:19 pm on June 8, 2008 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator robert_charlton is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2000
posts:12232
votes: 364


Just to let you folks know..

I posted uploaded another page... my lost page was re-indexed and ranked as before and the new page is ranking well..

David - Thanks for the follow-up. It sounds like this was just a situation of Google databases getting synched up.

It's very possible, even with established pages, to see two different rankings simultneously on different machines. There are constant adjustments being made at datacenters, and it takes time for data to migrate. That said, you may yet see the drop that others are discussing. ;)

Re the second page you uploaded... if it was substantially a dupe of the first page... or if it, say, had the same title... one of those pages is likely to disappear. You can target a phrase on more than one page, but if you use the same content on more than one url, you will have dupe content problems.

7:55 pm on June 8, 2008 (gmt 0)

Senior Member

joined:Jan 27, 2003
posts:2534
votes: 0


I notice that the original post was from the UK, and that's where I got most of my data from, so I don't know if there's a regional influence on this.

From the sample I took, around 60% dropped shortly after the initial appearance in results. A page with external links or significant internal links was significantly less likely to drop out, but this did not explain the whole relationship between the page and the chance of disappearing.

Note that I did not take a particularly large sample to obtain these numbers.

11:12 pm on June 8, 2008 (gmt 0)

Junior Member

10+ Year Member

joined:May 30, 2008
posts:47
votes: 0


update...

page 1 on (3rd June 2008) - indexed on 4th June, de-indexed on 5th June <-- still de-indexed
page 2 on (3rd June 2008)- indexed on 4th June, de-indexed on 5th June <-- sill de-indexed
page 3 on (4th June 2008)- indexed on 5th June, de-indexed on 6th June <-- re-indexed 8th June and regained same serp positions.

12:53 am on June 9, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


Hmm. This is odd. I am looking at a site newly online too.

A site:domain.com search shows 33 www pages. Clicking the omitted results link takes it up to 34 www pages.

A site:www.domain.com search shows 39 www pages. Clicking the omitted results link takes it up to 40 www pages.

A site:domain.com/ search shows 39 www pages. Clicking the omitted results link takes it up to 40 www pages.

The site only has www pages, and all links also point to www URLs too. There is a site-wide non-www to www 301 redirect also in place.

Additionally, there is a bit of variation depending on which Google IP addresses I select to do my search.

1:52 am on June 9, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


I've always assumed it was a way of making sure time-sensitive content appeared immediately

No doubt that Google does do this also - they watch queries for "burstiness" and take special effort with spidering and indexing urls with those terms. But I think that's only part of the picture. Another clue, on AOL site:example.com is usually a lower number of results for me than site:example.com/* on regular Google.

Part of that can be Safe Search - AOL defaults to "strict" whereas Google defaults to "moderate". But that alone is not the whole difference.

these pages are new topics so don't have as many internal links

That's a good observation. It might well be one of the factors involved with the indexed, de-indexed, reindexed cycle.

8:39 am on June 9, 2008 (gmt 0)

New User

10+ Year Member

joined:June 8, 2008
posts:33
votes: 0


Today I checked my test page again, it is now indexed however not even int he top 100,, when it first entered Google it was number 1.

What a strange world this is..

11:31 am on June 9, 2008 (gmt 0)

Junior Member

10+ Year Member

joined:May 30, 2008
posts:47
votes: 0


update...
page 1 on (3rd June 2008) - indexed on 4th June, de-indexed on 5th June <-- re-indexed 9th June with bad serp positions
page 2 on (3rd June 2008)- indexed on 4th June, de-indexed on 5th June <-- sill de-indexed
page 3 on (4th June 2008)- indexed on 5th June, de-indexed on 6th June <-- re-indexed 8th June and regained same serp positions.
12:22 pm on June 9, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:689
votes: 0


Getting de-indexed is absolutely normal in my experience. Has always happened, and then the pages come back in 2 days or more. I have seen this happen for 2 years or more...

However. This month has been nuts. Blogsearch pinging not working, sometimes working but pages not making it to the main index for a week.

Since yesterday though, a lot of never indexed and de-indexed pages seem to be on the way back.

9:02 pm on June 10, 2008 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 13, 2008
posts:60
votes: 0


topranker please could you verify if you are disappearing are your site is around #700 or #900 searching keywords in all different Googles's DCs ?
1:16 am on June 11, 2008 (gmt 0)

Junior Member

10+ Year Member

joined:May 30, 2008
posts:47
votes: 0


All seems ok to me with the keywords i've checked, also stats appear to be showing normal activity at around 8,000 to 10,000 a day.

Our site is uk targeted so doesn't do well in .com but ranks pretty high in .co.uk with site links even for generic keywords.

7:35 am on June 11, 2008 (gmt 0)

New User

10+ Year Member

joined:June 8, 2008
posts:33
votes: 0


This is weird.. yesterday my test page was raised to number 8, this morning it is gone again.. surely Google must be having some kind of hiccup?

David

11:27 am on June 11, 2008 (gmt 0)

Junior Member

10+ Year Member

joined:May 30, 2008
posts:47
votes: 0


Here is todays update on my three test pages

page 1 on (3rd June 2008)
- indexed on 4th June wirth very good rankings
- de-indexed on 5th June
- re-indexed 9th June with bad rankings
- 11th June ranking well for just one search term, rest poor

page 2 on (3rd June 2008)
- indexed on 4th June with bad rankings
- de-indexed on 5th June
- 11th June still de-indexed

page 3 on (4th June 2008)
- indexed on 5th June with bad rankings
- de-indexed on 6th June
- re-indexed 8th June and regained same bad rankings.
- 11th June now has very good rankings for many search terms

Pages 1 & 2 are relvant to each other so only link to each other.
Page 3 has relevant links from 4 other older indexed relevant pages

All pages are still linked from main page and an (all pages) links page.

11:45 am on June 11, 2008 (gmt 0)

Junior Member

10+ Year Member

joined:May 30, 2008
posts:47
votes: 0


Serps still seem very unstable, although our overall site stats are slightly down we have had some reports saying a lot of customer pages have died but still indexed which were previously very busy, so that must mean a also a lot of other pages must have increased in rankings to make up for the loss of traffic on those pages which have been hit.

We really only get to find out about poor performing pages and not pages getting busier.. people only tend to report things if they have a complaint.

It maybe that some pages have been penalized for some reason, but the overall site certainly has not been penalized as its still getting a lot of very good rankings for high traffic keyword terms.

7:42 am on June 13, 2008 (gmt 0)

New User

10+ Year Member

joined:June 8, 2008
posts:33
votes: 0


This morning one thing occured to me regarding my page.. now I use wordpress and am wondering am I guilty of duplicate content?

this is my post page which is fine..

http://www.example.com/category/page-title/

............................................
this is the page that the tag cloud links to

my page is duplicated here?

http://www.example.com/tag/page-title/

................................................

this is the category page.. my page is also displayed here

http://www.example.com/category/page-title/

..........................................................

I checked many popular blogs and to me it seems the same for them.. I am a little confused here?

Regards David

[edited by: tedster at 8:44 am (utc) on June 13, 2008]
[edit reason] switch to example.com [/edit]

11:48 pm on June 15, 2008 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 13, 2008
posts:60
votes: 0


ok, as I told in my previous post, I am investigating in disappearing positions of many sites and realizes the disappeared pages take positions or near to #700 or near #900 in all DCs.

Then some pages returns in its original ranking positions (or better), instead other pages remain down (also for months).

If checking ranking of your sites you'll find a similar situation, could you please post about it here? Thanks.

[edited by: tedster at 11:55 pm (utc) on June 15, 2008]

11:59 pm on June 15, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


misterjinx, have you read about the -950 penalty [webmasterworld.com]? That is what you're describing, and i'm glad tro hear that you do a thorough check. It is important to know whether a url from your site is completely gone from the index, or is still in the index but sent down to the nether regions for a term where it used to rank.
3:02 pm on June 19, 2008 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 13, 2008
posts:60
votes: 0


Yes tedster and you know (of course) I read it... the url is not completely gone.

A note: in other 2 cases pages completely disappear from index for 2 months (feb-mar '08) to appear again in april 08 ranking around #3-#5. Now lower ranking around #13 but it was for an Italian SEO contest so it is lesser interesting for me now.

4:16 pm on June 19, 2008 (gmt 0)

New User

10+ Year Member

joined:June 8, 2008
posts: 33
votes: 0


My new site I made the test page for is ranking 1 [Yahoo,Alta vista,Aol] for most terms I target pages for ..

I have about 130 pages now, 20 indexed by Google, and a few for Yahoo....

Right now it looks like if I build a page aimed at "Blue Widgets" I can get to the top of Yahoo but not Google....

Weird thing is..

However... if I "mention" the word Widgets on the Home page the home page gets ranked for the term and not the page that was built for "widgets"....
A while back it seemed if you ranked for something in Yahoo you would have a similar ranking in Google and vice versa, Google UK now though seems to be doing some real crazy stuff... in a way I hope it is broken and that there is a chance they will fix it...

R.e toprankers last post.. my home page is the only pages with links going to it.. [ got some links today for the internals ]

Regards David

David

10:38 pm on Aug 13, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 17, 2004
posts:1354
votes: 0


<moved from another location>

Hello,

My site was dropped from the index only two days after being crawled and indexed. How can this happen? My site plays by the rules; no cloaking, no spamming, no hidden stuff, no adware, nothing. The only backlinks I have (yet) is from some social bookmarking sites used to get the word out. I do have some affiliate links (it's an aff. site), but I also have several pages of unique, well-written and relevant content.

I don't understand!

[edited by: Robert_Charlton at 11:46 pm (utc) on Aug. 13, 2008]

12:16 pm on Aug 16, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 17, 2004
posts:1354
votes: 0


It appears I have been re-index and am ranking as good as before. Maybe it's just a matter of different datacenters having to synchronize with some master server?
This 51 message thread spans 2 pages: 51