Forum Moderators: open

Message Too Old, No Replies

About this crawl, if a site is down when the spider

was in action, is there still a chance.

         

zeus

2:28 pm on Jan 5, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If a site is down or just a blank site for one day and you know the spider is in action is there still chances to get the page spidered, I think when some sites link to that page then the spider will update the site right even if it has been there a day ago and the site was blank.

Im not totaly sure on this.

zeus

vitaplease

7:16 am on Jan 6, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I think this is the nightmare of every webmaster.

If you have a high enough Pagerank, Google will revisit your site for sure if you have a day off.

also check this thread on Google "...boosting the retry attempts..."

[webmasterworld.com...]

percentages

8:04 am on Jan 6, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



zeus,

In my experience Google is very tollerant of sites that are down when they try to crawl.

These days most of my sites have been moved to dedicated hosting that is very reliable, but I still have a few on shared and less reliable hosting waiting to be moved.

These sites have been down on occassions when Googlebot has visited all the others (so I assume it tried those also), but, Google has not dealt them unfair blows because of it. Generally the sites have come back within a few hours and Googlebot came back and found them :)

If your site is down for days when the deepcrawl bot is out then you may have problems the following month. But if you expect to make money from the site why use a host that can't keep the servers up for 99.99% of the time?

zeus

12:53 pm on Jan 6, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



vitaplease

The problem was that on my main site, there just was a blank site (white) and nothing els, so not a 404, so I hope it will come back trough other links to my site I do have a PR6 on this one.

The other page I by mistake uploaded a different index page to that server which have nothing to do with that site.

All that I saw the next day, then I changed it.

thanks for the reply

zeus

ciml

6:07 pm on Jan 6, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



zeus, recently I've found that the deep Googlebot visits pages about twice per month even if it can get through. Hopefuly, you'll be OK.

zeus

8:12 pm on Jan 6, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



ciml, thanks for your support, I do get a new cache-date on my main site like today its 4dec. every other day I hope that they use the latetst everytime they rank the pages in the update.

I just hope for the best, all the best to all of you.

zeus