Site not spidered in two months

Google claims my site is not penalized. But it is

[leifwessman]

My website (PR 5) has not been spidered for two months. Before that it was spidered every four och fifth day. I'm logging all the spider activity so I'm 100% sure. Also, when I look at the cache of my website I can see that it is from january 12 2002.

I've written to Google about it and the response was:
"Thanks for your email. Your site has not been penalized by us. Your site is in our index"

But since GoogleBot refuses to visit my site - I'm sure that my site is penalized for having repeated keywords (although keywords deleted long time ago but still appear in the Google Cache).

Any ideas what's going on? Could Google be wrong?

Leif

[TheDave]

You still have a PR5? If you still have a PR5 then there no reason to believe you have been banned just because the bot hasnt visited. Maybe you need to freshen some content on the home page to get the bot interested again.

[leifwessman]

The content is the same as before. Only new toplists and tips are added (changed every week).

To get the bot interested I would at least need one visit... I'm getting none.

Leif

[leifwessman]

Yes, I still have PR 5.

[kwngian]

leifwessman

Could it be your inbound links are insufficient. The bot is very busy you know.

Maybe you could try and get more inbound links especially on highly active sites - guestbook maybe? just one won't hurt will it?

kwngian

[Tony_Perry]

Add new content (fresh and relevant) and new links (high quality)and you will be ok.

[creative craig]

Just hunt for the links and the bots will come ;)

Craig

[heini]

Good tips, but leaves us with some questions. How should Google even know the site in question didn't change when the bot didn't check it?
Assuming it were true that Google did not come because the site didn't get new links, then this would be a pattern of behaviour new to me. Even old abandoned sites got regular crawls, at least if they still have reasonable PR.
Not saying it's not true, but would be news to me.

Leif, did you check your logs manuallly? Does your site get spidered by other bots? Your robots.txt okay?

[leifwessman]

Leif, did you check your logs manuallly? Does your site get spidered by other bots? Your robots.txt okay?

No, all visits from GoogleBot is logged to a specific logfile.

My site is spidered from other bots, yes.

I have no robots.txt file.

I have plenty of inbound links - even more now than before (january)

Leif

[rogerd]

The answer is that yes, Google can provide an cut & paste response that your site is not penalized when it is. (Unfortunately, I don't think there's a central database that can be easily checked for penalties related to any and all infractions. No inside info, just a guess based on snippets of comments here and personal experience.) I'm sure Google gets lots of mail from people who can't imagine that their site is #1, and ask why they are being penalized. The low-level techs who reply to these get in such a groove, it seems, that sometimes sites that really have been penalized get a response like yours.

Having said that, however, with the available data I see no reason to believe that your PR5 site is carrying a penalty. Penalties usually seem to take the form of PR reduction rather than constant PR with reduced spidering. As others have suggested, keep the content and links coming and things should sort themselves out.

[borisbaloney]

As others have suggested, keep the content and links coming and things should sort themselves out.

Rogerd you are clearly experienced with SEO and I am only a newbie, but to suggest waiting longer when he has already waited two months without any sign of googlebot seems a little off to me.

Considering the cached version is extremely old as well, it is clearly having a big negative influence on what is probably a commercial site. Imagine putting in months of hard work for no reward from your primary source of new visitors, and no signs that things will change.

If it was me I would hope that the google staff handling email have fallen in to the cut and paste routine as rogerd mentioned. Keep persisting sending clear, consise, and detailed emails to google then hope you strike a tech willing to investigate further.

Good luck.

[heini]

I don't think the cache is such a huge factor for the success of an ecommerce site.
Basically, if the ranking is not negatively influenced, and the PR is unchanged, I don't see a reason to be overly alarmed.
Another possibility to check for would be a dns change.

[leifwessman]

Its not the cache that is the real problem. The real problem is that Google doesn't find the links to new pages.

Leif

[borisbaloney]

I didn't mean to imply that the cache on its own its a huge problem. I think it is an indication though that everything is not normal with the site's listing in google.

My _guess_ at the problem is that it is not a penalty. I would suspect a problem on the host (surely couldn't be a firewall rule, could it?), or a glitch at google.

Even by simply waiting and hoping googlebot comes back falls into the old tech saying - problems that go away by themselves tend to come back by themselves. Find the issue and resolve it.

[heini]

>Find the issue and resolve it

Yes. Exactly what I'm thinking. There's no indication of any penalties from what Leif says, so I'd be looking into the reporting issue, and in the general accessability.

[leifwessman]

There is no problem with the website. It has validated HTML and more than 2000 visitors each day. There has not been any major HTML changes for a long long time.

[MetropolisRobot]

I also have a site that is the same with regard to having not been visited for a while.

Now originally this site did get connected to some bad neighborhoods by some links that I neither wanted nor solicited.

With Google the site is a Pr3 and has not moved from that in 2 months, BUT neither has it been blown out of the index either. I just get the feeling that Google is busy, and that if it has time it'll get to my site.

I am trying to encourage it with the usual moves (links, changes, new content), but at the end of the day my business is not dependent on Google for its success and therefore I don't lose too much sleep over it. People still come to it.

[takagi]

Hello Leif,

Three small questions:

1. Do you have a lot of dynamic pages on your site? Think about this situation. A *.php or *.asp etc file is not changed. But the data in your database does change. If G checks the date the php file is last changed, the server sends the old date to G, not reflecting the changes to your database. I read some time ago on WW about this problem.

2. Do you have an 'Expires' meta in your files?

3. Can you try to get an outside link to one of your new pages? If this link is found by G and this page is spidered or not, it can help you to understand what is going wrong.

[heini]

Are those by any chance dynamic sites? Just wild guesswork.

[leifwessman]

Do you think that this may be the cause?

This is how my headers used to look:

hdr>HTTP/1.0 200 OK
hdr>Server: Apache/1.3.26 (Unix) AuthMySQL/2.20 PHP/4.1.2 mod_gzip/1.3.19.1a mod_ssl/2.8.9 OpenSSL/0.9.6g
hdr>Expires: Wed, 12 Mar 2003 19:49:54 GMT
hdr>Date: Thu, 13 Mar 2003 19:49:54 GMT
hdr>Pragma: no-cache
hdr>Cache-control: no-cache,no-store,must-revalidate
hdr>Content-Type: text/html; charset=ISO-8859-1

I removed the following headers some time ago (don't know the date):

hdr>Pragma: no-cache
hdr>Cache-control: no-cache,no-store,must-revalidate

However, since I'm logging every request I should have noticed if GoogleBot did a request. Even though they were just looking at the headers, right? Or: Is it possible that GoogleBot just du a HEAD to every page before it fetches it - at a later time? But doesn't HEAD makes the server to generate the whole page? I'm confused.

Leif

[leifwessman]

I read some time ago on WW about this problem.

How can I find that thread?

Leif

[takagi]

The thread I was looking for, I don't see anymore. Maybe these 2 threads will help:

[webmasterworld.com...]
[webmasterworld.com...]