Googlebot is crawling my site everyday

[Nitin]

Is it possible that Googlebot is crawling my site regularly,
I am checking my server log files for one month regularly,
To my surprise almost everyday googlebot is spidering my site.Please let me know is it Googlebot or something else
I am forwarding some contents of my server logs

Google (5:43 4/22/103) www.sitename.com/policy.html Googlebot/2.1 (+http://www.googlebot.com/bot.html) 64.68.82.6

Google (5:43 4/23/103) www.sitename.com/email.html Googlebot/2.1 (+http://www.googlebot.com/bot.html) 64.68.82.67

I am new to this field,sorry for stupid question.

Thanks for youe time.

[johannes]

Yes, you are beeing spidered by Google. Those are the freshbots, they come every day to me too, read more by searching for freshbot. The deepcrawling bots from Google comes around once a month.

It probably means that you have a pretty high pagerank and/or updates you content once in a while...

[guddu]

Do you mean to say, if there are frequent updates say on a weekly or fortnightly basis, then chances for Google crawls increases. What if the updates are on mothly, bi-monthly basis or if there are no updates for say 3-4 months?

I have a client with a static site and he didn't update it for 3 months time. However, some new incoming links were added to the website. But now, when it is being updated regularly on a monthly basis since Dec, 2002, no fresh listings have appeared in Google. However, the new links are being shown.

Should we expect a fresh listing in next Google update.

The website is on yahoo geocities (free website using the subdomain) and we do not have access to logs. Can you suggest some measures to track Googlebot crawls.

Regards

Guddu

[johannes]

Actually I'm not at all sure that frequent updates increases the chance of freshbots coming around. But it sounds logical to me. There should be information about this in older threads.

What I am sure of, is that a higher pagerank means the freshbots are more likely to show up.

guddu: It should be possible to track the robots, if you can use PHP or CGI at geocities.

[guddu]

Thanks for such a prompt reply. I too do not know if geocities allow use of PHP or CGI.

I hope someone else at the forum would be able to help answering the questions.

Guddu

[Unversed]

If you are limited to a free host, and you need CGI, try Tripod

[guddu]

Thanks

But do you know if geocities allow php or cgi or even asp scripting?

Guddu

[guddu]

OK I got the answer. It allows PHP.

refernce:

[us.geocities.yahoo.com...]

Now can you please help me get the code for tracking googlebot in php

and where and how to incorporate the file on the directory at geocities. Is this have to be linked by any HTML page of the website.

Regards

Guddu

[johannes]

You probably wont get a finished and all ready script here guddu.

Try [php.net...]

Or try posting in the PHP and scripting forum somewhere here at WW.

[guddu]

Thanks Johannes.

[power_iq]

The Deepcrawler doesn't visit my Website since starting the Crawl.
hmmm, the Freshbot comes nearly every day. Could this be! Why the DC needs so a long time for?