Forum Moderators: open
Reading through prior posts in the fourum by Brett & Others, I came to know that we ccan track Deepbot & Freshbot crawl through the ip addresses 216.* and 64.* in the log reports.
I did the same, but got around 137 clicks through 216.* and even more by 64.* to my website in the log reports for last seven days.
Though, the website is being updated in Google for freshcrawl hits, regularly in April, the last update date being shown as 5th, April, 2003 in Google (till now).
However, no fresh pages added and no changes appearing in the website pages shown. (Just the homepage being shown as updated. )
I have a feel, that there is something wrong with the log analysis through 216.* and 64.* tracking.
Can anyone please help me get accurate report for Googlebot crawl by providing a specific URL / IP address for Deepbot and Freshbot.
Also, I read about ****, can you specify, whether it is safe to use i.e., not penalized by Google. (Is it Google's own website?)
freshBot: 64.68.82.* bah, listed for only a few days, short dinner date then it dumps you.
deepcrawler: 216.239.46.* Good bot, it likes you, listed until death do you depart. So don't make him mad or it will divorce your site. You can divorce the Googlebot by using your robot.txt file. It's much cheaper and faster than going to the courts.
Now if you use programs to submit your site, then some search engines might penalise you, though I don't know if Google would for using programs.
:::Sorry for posting the URL but how could I explain which site I wanted to ask for?
This is why URLs arn't allowed. That site is a Tool Site.
****
No Tools sites or urls please The explosion of Google related free and commercial site tools has opened a flood gate of url dropping to "tools" sites. Therefore, we request you don't post links to "Google tools" (there are dozens if not hundreds of them now as a result of the Google API).
****