Welcome to WebmasterWorld Guest from 54.159.89.7

Forum Moderators: open

Message Too Old, No Replies

The bot, the logs, and the update

identifying logs and the google bot

     
2:58 am on Sep 6, 2002 (gmt 0)

New User

10+ Year Member

joined:Aug 31, 2002
posts:39
votes: 0


Okay question:
Last dance i began seeing google in the logs a few days prior. One or two bites a day for a few days. Around the 28th
google started the deep crawl. My logs are still showing activity from the bot since, about a dozen or so pages per day.

Theory:
Google starts crawling the higher pr sites first in the dance. Mine is a pr5, so I am guessing I fall in the middle of the dance
somewhere. Are these small bites I am seeing prior to our deep crawl the effect of google crawling higher pr sites with
inbound links to us, and after our crawl from inbound links of lower pr sites? -correct me if I am wrong.

Also for 4 months running now I have noticed an unknown bot in the logs exactly a day before google shows up. Is this a
coincidence, or has anyone else noticed this? I gave it up as coincidence until this past dance...thought there may be more to
it. Now when I see the unknown bot show, I set out the snacks for google.

3:06 am on Sept 6, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member marcia is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Sept 29, 2000
posts:12095
votes: 0


Welcome to WebmasterWorld, argusdesigns. I've got one that's a low PR5 and it seems to get later than anyone else.

This strange bot, do you happen to have an IP number for it? If so, is it consistently the same?

3:36 am on Sept 6, 2002 (gmt 0)

New User

10+ Year Member

joined:Aug 31, 2002
posts:39
votes: 0


Hi marcia, thanks for the welcome.
As far as an IP #, no. But have since installed new web logs that track IP's so I will keep you posted on the results come the next dance. I dub the little critter "her shadow".

Thanks for the verify on the pr5.. sounds logical.

5:54 am on Sept 6, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 28, 2002
posts:79
votes: 0


more details about unknown bot? Maybe a rudimentary cloak checker?
6:20 am on Sept 6, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member rfgdxm1 is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 12, 2002
posts:4479
votes: 0


>I've got one that's a low PR5 and it seems to get later than anyone else.

Precisely how do you know it is a low PR5 and not a high PR5?

12:29 pm on Sept 6, 2002 (gmt 0)

New User

10+ Year Member

joined:Aug 31, 2002
posts:39
votes: 0


I am guessing that a low pr5 toggles between pr4-pr5 during the update. A high pr5 toggles between pr5-pr6 during the
update. I have heard people refer to different echelons of pr, but have not been able to really find any more info. on it.

more details about unknown bot? Maybe a rudimentary cloak checker?
Not sure about that one, we don't use any sort of cloaking or .asp files, just good ol' fashion html. It has occured to me that this may be the old log software, but i doubt it. I'll know more next dance.
12:47 pm on Sept 6, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 6, 2001
posts:2213
votes: 0


Maybe a rudimentary cloak checker?

Hello Arg....

I think Savvy was saying that it might be a check to see if you are using a cloak of some sort. Cloaking relies on the software being able to recognise a particular search engine and depending on who requests the page different versions are presented

EG
SE = optimised content, keyword rich
User = flash version or similar

Engines know what goes on so they look for sites that serve different pages to different IP addresses, so by using new IP's SE can hopefull (they think) tell the difference.

Cheers

1:33 pm on Sept 6, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member rfgdxm1 is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 12, 2002
posts:4479
votes: 0


Right, ukgimp. If Google wants to catch cloaking, send a bot not identifying as Googlebot, and from IPs Google doesn't use, and compare what is sent to the bot to what Googlebot got. This should easily catch cloakers. Googlebot seems to do a LOT of crawling. Google obviously has the resources to have secret bots running looking for cloakers.
3:01 pm on Sept 6, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 28, 2002
posts:79
votes: 0


Yes, exactly what ukgimp said :) (Thanks ukgimp)
I did not mean to imply that you were cloaking or anything about your site in particular. :)
5:50 pm on Sept 6, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Sept 4, 2002
posts:136
votes: 0


I have been listed in google for a little while now, but since my initial listing I have not seen an update. I was under the impression that google does a sweep once a month. Am I correct in assuming this? I have a PR of 4 and was wondering if this is the reason my site's not getting updated