homepage Welcome to WebmasterWorld Guest from 107.20.131.154
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Accredited PayPal World Seller

Visit PubCon.com
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
Pages Dropped or maybe not indexed?
Only Root is Indexed
CelticFC




msg:73489
 11:56 am on Dec 22, 2003 (gmt 0)

Hi,

The URL i submitted to Google has now been listed (only this morning) and i have received a good ranking for probably the sites most important keyword. The trouble is Google only appears to have listed the root URL.

I have tried utilities to check that the site can be indexed correctly with search engines. Does it take time for the rest of the pages to show up on Google? or perhaps the pages have been dropped?

I did not use the robots.txt file as i was under the impression that if the file did not exsist then the spider was free to index the full site is this correct?

Any help or advice anyone can give me would be greatly appreciated.

Thanks!

 

dcrombie




msg:73490
 1:18 pm on Dec 22, 2003 (gmt 0)

It's usually just a matter of waiting. You might want to check that your HTML is valid and that the site is accessible to search-engine spiders (ie. doesn't rely on Flash or JavaScript for navigation).

nileshkurhade




msg:73491
 6:39 pm on Dec 22, 2003 (gmt 0)

CelticFC :

The trouble is Google only appears to have listed the root URL.

I think this might be the FreshBot so your site may disappear for a few days, but dont worry the DeepBot will eventually index your site and if your SEO is good you will have good rankings.

[edited by: nileshkurhade at 6:42 pm (utc) on Dec. 22, 2003]

nileshkurhade




msg:73492
 6:40 pm on Dec 22, 2003 (gmt 0)

CelticFC

And let me add Welcome to the Forum.

superscript




msg:73493
 6:45 pm on Dec 22, 2003 (gmt 0)

Hi CelticFC, (any relation to NFFC? ;)

Some people think a robots.txt might be desirable, but you're quite right that if you don't have one, it means 'go ahead and crawl', as robots.txt is designed to exclude. My main site didn't have one for years and were listed fine. Personally I'd be pleased to get your index page in, if it sticks it's a great start!

webdude




msg:73494
 7:01 pm on Dec 22, 2003 (gmt 0)

Just wait, it will come. I use a blank robot.txt and have no problems.

BTW Welcome to the board CelticFC!

CelticFC




msg:73495
 7:10 pm on Dec 22, 2003 (gmt 0)

Thanks for the advice everyone!

Just need to sit tight and see what happens!

Any idea how long it normally takes for the bot that digs deeper to drop by?

Thanks Again!

div01




msg:73496
 7:30 pm on Dec 22, 2003 (gmt 0)

Usually a month...but I have one site that's been waiting for a good deep crawl since May! :(

superscript




msg:73497
 7:45 pm on Dec 22, 2003 (gmt 0)

div01,

I'm relieved to hear from someone else in the same boat - my site hasn't been crawled properly for months. The index page gets a crawl every couple of days. It's a high PR site, and only about 100 pages - I just don't get it :(

div01




msg:73498
 8:24 pm on Dec 22, 2003 (gmt 0)

What makes it worse it that mine is just a plain static site with about 100 pages, good navigation, no js, and a sitemap. Strange really...main page PR has now dropped to 3, sub pages at PR0, and I am about ready to send the dreaded "reinclusion request" for possible help from G.

bob742




msg:73499
 8:57 am on Dec 23, 2003 (gmt 0)

We had a site that only had home page (PR5) indexed. Last month we added another domain as an alias, and google has just finished indexing about 5k pages under the new domain name.

My conjecture is that the database record for the original domain was corrupted.

superscript




msg:73500
 9:29 am on Dec 23, 2003 (gmt 0)

Hi Bob,

Last month we added another domain as an alias, and google has just finished indexing about 5k pages under the new domain name.

Can you expand on what you did with the alias?

fom2001uk




msg:73501
 9:47 am on Dec 23, 2003 (gmt 0)

Welcome CelticFC.

Like the handle. Here's to another league title :-)

simonuk




msg:73502
 10:56 am on Dec 23, 2003 (gmt 0)

I think the deep crawl bot has the flu and is only working part time while it recovers. Or maybe too many Christmas parties. I think I'll email the DC bot the link for this new hangover cure sweeping America :)

The DC bot has been acting a lot stranger the last 3 or 4 months. I would get a regular deep crawl on all my sites quite often but now it varies greatly which site gets deep crawled if at all.

It doesn't seem as active as it used to be.

Simon.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved