Welcome to WebmasterWorld Guest from 54.159.250.110

Forum Moderators: IanTurner & engine

Message Too Old, No Replies

Google missing old pages in search

Bing and Yahoo find them

   
11:31 pm on Jun 20, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I have three posts on my website that mention an author's name, but when I search Google UK for the name, only one of the posts is found. The posts date from 2004, 2005 and 2006.

I added a test page with the name in it yesterday and funnily enough Google indexed it 20 hours ago, so it now shows up. But I cannot understand why such old posts are not being indexed. (Other words in them can't be found in a search either.)

All my posts use the same two link formats and are clearly linked from other pages in lists.

I also tried Bing and Yahoo, both of which found all three posts.

Why would Google have missed two of the posts? And is there anything I can do to get them indexed?
2:15 pm on Jun 21, 2010 (gmt 0)

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



I don't know anyone will know for certain why they are not there.

Here's what i'd question.
1. Could there be a block on google indexing the posts?
2. Could your site or those pages have been down when google came to spider it?
3. What are the links to the posts that enable googlebot to follow it to the posts?

Google likes new 'fresh' material. It will index old material, but, sometimes, it takes what seems like an age to index it. Sometimes years.

The fact that it's old material will put it into that category that it's not 'fresh' content, which is what google is hot to index.

HTH
10:03 pm on Jun 21, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thanks for the reply. To answer your questions:

1. If there is I don't know why

2. Good point. But surely G would have returned after so many years?

3. Two archives pages listing all the posts, plus category pages relevant to the posts.

I've also added a search form which can find the posts.

> Google likes new 'fresh' material.

The posts were new when they were created, so G should have caught them.

Extra: I just tried a search in Ask Jeeves and it only found ONE of the posts. Hmmm...
5:59 pm on Jun 24, 2010 (gmt 0)

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



1. If there is I don't know why


check your robots.txt, or meta tag for <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

However, if it's in other search services, that makes the meta issue unlikely.

2. Good point. But surely G would have returned after so many years?


It should have, but if it didn't find links, it won't necessarily find it. If it's a deep link, it'll take longer to find, too.

3. Two archives pages listing all the posts, plus category pages relevant to the posts.
I've also added a search form which can find the posts.


Check your site stats for spidering.

The posts were new when they were created, so G
should have caught them.


True, however, unless it indexed it in the first place, then dropped it, you'll need to look at another way. The file may have been unmodified and googlebot is looking for fresh material, as I mentioned. You could also try refreshing the file, too so that it has a new file date on the server.