Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

Googlebot dilemma

Crawling hard - Site not found

7:20 pm on Dec 4, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 23, 2003
votes: 0

Googlebots (the moz compatible ones) have been hitting a site of mine VERY hard for the last two weeks (13,000 pages in 5 hours yesterday, 10,000 in 3 hours today), this site is, and has been missing from the index since August 2004. site:site.co.uk produces no results, but allinurl:site.co.uk produces 21,500 pages all of which appear to be the dreaded pagejacker sites, with clickclocker.php in their urls.

Should I ban googlebot from the site, or should I allow it, in the hope that whatever caused/is causing the sites demise, will be rectified by google, sooner rather than later? (bandwidth isn't an issue, but other websites on the same server are suffering degraded performance whilst this heavy crawling is going on)