Welcome to WebmasterWorld Guest from 126.96.36.199
Forum Moderators: open
crawler10.googlebot.com - - [07/Apr/2003:10:57:20 -0700] "GET /1Crim.html HTTP/1.0" 301 258 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
crawler10.googlebot.com - - [08/Apr/2003:02:11:12 -0700] "GET /robots.txt HTTP/1.0" 200 188 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
crawler10.googlebot.com - - [08/Apr/2003:02:11:13 -0700] "GET / HTTP/1.0" 200 - "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
These are the only two requests.
Just a thought.....
It drove me crazy - one of my sites had a complete re-design days before the last update and was getting fresh results in www2 and www3 but when the update had completed freshbot was not going so deep and the results never showed in www.
I'm new to this game. How can I check to see if the freshbot is visiting my site?
The two posts just below yours contain two examples of googlebot UA (User Authorization) strings.
To check them (UA strings) you'll need to be able to read your access_log files. Do you currently have access to them? Check with your host server to see just what you are entitled to use.
Access_Log Files house every impression every visitor (human or otherwise) makes.
Some stats programs/application either provided by your host server or yourself (in certain cases) show similar data - but not as detailed as those you can inspect yourself via access_log files.
Doing a Site Search [webmasterworld.com] using 'access_log files' pulls up some additional readings.
access_log.3:188.8.131.52 - - [20/Mar/2003:19:11:25 +0100] "GET /zblue/index.htm HTTP/1.0" 200 10441 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Googlebot crawled 144 pages ... how long do they take to include it in the database? .. Do I have to expect another crawl this month before I'm included in the search database?