From Fully to Partially Indexed -

what happened?

[djsyc]

I have a site that is fairly old and I recently added a new directory that contained information and product specs - each page had a brief description of a product line and also contained some company specific about us info that was copied across all the pages - the intention was that most visitors would never be going to more than one informational page and would more likely go to our online store.

These pages were added to the index FAST - within a couple weeks, and were beginning to rank well for their particular keywords.

A colleague did an update to the main index a couple weeks ago - the only domain level static link to the new directory was removed because he broke the template and forgot to update the links.

The site was reindexed a couple days later sans the link - thus the new directory was no longer connected to the main site.

We caught the issue and checked - our new directory still existed in the index, but within a couple days our new directory and all of the pages disappeared from the index - all pages are "partially indexed" instead of fully - which they were before.

I am in a quandry..we have repaired the link, and hopefully googlebot will follow and reindex the sitemap and other pages..

But I am afraid that we might have had too much duplicate content page to page - causing the removal from the index. The main part of the site that was older still remains - its just the new stuff that is gone.

Should I rebuild the informational pages and remove the duplicate content (about us, company info and general info about our products and services) from each page, or should I let the page sit?

Is it possible to drop off the index if the server is down during a bot visit?

Im perplexed. Any suggestions?

Me

[awebguy]

If the piece of text you added to all new pages is significantly big, gooogle penalize you for dupe content. Put it as a java script or similar to prevent this text from indexing.
It will be visible, but will not disturb gbot.

[Ept103]

awebguy, I'm looking to setup something like this as well. I'm not sure how to do it.. How would one go about putting text as java script?

[awebguy]

Ept103, learn dynamic content approach

[djsyc]

So...

Any idea on how long it takes for google to look at your stuff again? Any other possible reasons to go from fully indexed to partially indexed?

I Notice googlebot will occasionally crawl - and even index - a sitemap or links page - but it will take months for googlebot to actually ever bother following links. Any idea why?

me™

[Ept103]

awebguy, I understand dynamic content.. I just don't understand how you are making text in such a way that you can hide it from googlebot through javascript. Are you blocking an included javascript file from robots.txt?

[awebguy]

I have js folder blocked for all bots in robot.txt
My JS texts have nothing common with my main content and I use them only for advertising.

[MHes]

djsyc

Try getting some deep links pointing at the new pages from other sites with reasonable pr.... pr3 or more. This may get the spider interested.

Also you could put a few static deep links from your home page deep into the new pages.

With your new pages, make sure title tags are different for each page and that each page links back to the home page. Also throw in a few links from one new page to another where you relevantly can, but in a 'unpredictable' way. Try and put in as many unique bits of text and links for each page, even the odd link out to another site, the purpose being to avoid any 'dynamic' page pattern across all the new pages.

Lastly, get some new links in to your home page. You may not have enough pr to get the spider bother going deep into your site and take the new pages seriously.

[djsyc]

thanks for the advice, that was the route I have been considering. Thanks for the info!

me™