Forum Moderators: Robert Charlton & goodroi
I have 4 unrelated sites that all fell off Google back in January all at the same time. These sites are now all in the supplemental index, meaning that they get no traffic. The only way to find them is to search by domain name, and this just displays the domain names without title or description. Here is a typical case from one of my sites:
If I search for: ‘www. mysite .com’
I just see just the listing for my url with no title or description
If I search for: ‘site:www. mysite .com’
My index page is missing. 50% of the pages from my site show title and description, and the other 50% just show their url. If I click cached of one of the pages with title and description the result says: ‘did not match any documents’.
If I search for: ‘allinurl: www. mysite .com’
I see no index page, but I do see 2 pages from my site. The cashed shows that these pages were retrieved on Mar 20, 2005.
Some facts about my sites:
1.All 4 sites disappeared from Google at the same time back in January.
2.All 4 sites use shared hosting with a big (cheap) domain register with a silly name! (I am sure you can guess who I mean).
3.All four sites shared the same name server IP address block.
4.All four sites are still spidered by Google but very infrequently now.
5.My hosting company admitted that they had a domain name server problem in January. However they were not very specific about details.
6.All four sites are PR5 or higher, are about 3 years old and were very well positioned for their relative keywords.
Questions:
Have my sites been penalized for some reason? The first time I emailed Google, their responses indicated that there had been a spidering problem. Basically they said the Google index contains two types of pages: fully indexed and partially indexed pages, and that my page is currently partially indexed. Because their robots were unable to completely review its content during our last crawl, my site appears without a cached copy or detailed title. Instead, it's listed by its URL.
Six weeks later I contacted Google a second time and I got a more cryptic response. They said that they are unable to send personal responses to all of the requests they receive to review individual website content. They said that Websites can fall out of our index for many reasons, including penalization. Certain actions such as buying or selling links to increase a site's Page Rank value or cloaking - writing text in such a way that it can be seen by search engines but not by users - can result in penalization.
My sites comply 100% with googles guideline. I have not engaged in buying links, cloaking, or link exchanges. All my link exchanges are with appropriate similar theme sites as my own, and in fact I have not exchanges links with anyone for at least six months.
Any guesses, advice, or opinions you could share would be most appreciated.
Because their robots were unable to completely review its content during our last crawl, my site appears without a cached copy or detailed title
I'd try to determine why they were unable to spider your site.
Check to make sure you don't have a robots.txt file excluding the spider.
Was the server down at the time the spider came by?
Try pressing your hosting company to get more information on exactly what their domain name server problem was and if anyone else hosted on their servers has experienced the same problem.
Yes my hosting company is GD. Have others had the similar problems that you know of? Would you propose I switch to another host that has a more serious reputation?
Bobby
Nothing wrong with my robots.txt. GD is not that helpfully giving specifics or even understanding the problem as I describe it to them.
if so, i am on GD and had no problems.
I only mentioned that my four sites shared the same IP block to illustrate that if I had a name server glitch it may have affected all four sites. I am not fully convinced that dedicated IP make much difference since 97% of all website use shared IP.
I did however about 4 weeks ago set 3 of these sites missing from Google to dedicated IP’s. Since then I have noticed absolutely no difference. Google is not spidering them any more than before, only about once every 7-10 days.
I am also facing the same problem. I have 5 sites which are interlinked with each other.
My rankings disappeared from google in January.
They all are at same hosting plan and sharing the same IP....
Index Pages of my sites are not crawling at all & showing without Title/Description...Though internal pages are crawling weekly.
Did you find any solution of this problem?
Solution - change to another hosting company that offers a unique IP.
When doing a site:www.mysite.com search, the following happens.
1. It used to have 1500 pages indexed now it says it has 800. There are now over 2000 pages.
2. An increasing number of the pages listed are URL only. When searching for exact strings of text from these pages they are not returned even when there are no results to show (i.e. it's not a duplicate content thing).
3. My index page and many of the other high up pages only show up when &filter=0 is added to the string (include omitted results). It seems these pages are the ones with incoming external links. It could be the 302 issue. These pages do show up for exact phrases and occasionally for detailed search queries.
4. When searching for "Company Name" (yes quotes) I do not show up in the first 1000 results.
Background- Site was originally part of another site. The subdirectory of the old domain was 301'd to the new domain in June of 2004. The vast majority of the links have been updated and I recently (about 1 month ago) removed the 301's. The pages have never ranked at any time since moving to the new domain.
Unfortunately, Google couldn't get the old .html files out of its listing for a long long time (some of them were commentary files dating back to 2000), and the new .shtml files were not being spidered by the site. Some still haven't been, nearly one year later (I did this in approx. August 2004).
One way I managed to get some of the best of the older commentary pages listed was to either link to them singularly from new commentary pages (which, from my main page, were being spotted right away). Another was manually submitting the "lost" classic commentary pages one at a time as the new .shtml files; a third was to create an .html file for those still in Google and make it an auto-forward to the new .shtml file.
Any way you slice it, it's been a pain to deal with, and the other major web browsing sites out there have been able to catch up without too much trouble.
What's the lesson? If you're going to change your extensions en masse for an entire page, keep in mind that you might pay a serious price for the move. Unless you have something set up to forward every old .html ending page to the corresponding .shtml page (which I didn't have easily available to me at the time).
FWIW, my main page (without any .html or .shtml ending, just www.*****.com, has a PR4 from Google) has never had a problem appearing on Google. But if you tinker with the sub-pages, you might run into problems.
On the GD hosting thing, AFAIK they had blocked Googlebot from some of their servers, and sites were also slowly falling out of the G index.
