Checklist for low spidering

Main points to visit when you have this issue

[designhaus]

Hi all ... I have recently taken over the maintenance of two new sites and they were redesigned and moved onto a new server. The issue I am having is that Google is simply not spidering the sites after the server move. Previously they were spidered daily and on a big scale.

We have over 50 sites on the server and two of them in question who are big blue chip companies are getting visited by Google once a week whilst other smaller sites on the server which have NO updates at all get loads of visits from the Google bot. So I had to ask myself WHY!?

Here is a checklist for anybody else who may suffer a similar problem:

1) We have a firewall on the server
2) The sites robots.txt file in messed up
3) The server ip has been assoaciated with spam
4) The server company is blocking spiders

Now these are the main ones we could think of and I would appreciated any further points to be added.

[collymellon]

You could submit the site map for Google to come along and spider it, also have you tried removing the robots.txt?