Welcome to WebmasterWorld Guest from 54.226.246.160

Forum Moderators: open

Message Too Old, No Replies

The bot, the logs, and the update

identifying logs and the google bot

     

argusdesigns

2:58 am on Sep 6, 2002 (gmt 0)

10+ Year Member



Okay question:
Last dance i began seeing google in the logs a few days prior. One or two bites a day for a few days. Around the 28th
google started the deep crawl. My logs are still showing activity from the bot since, about a dozen or so pages per day.

Theory:
Google starts crawling the higher pr sites first in the dance. Mine is a pr5, so I am guessing I fall in the middle of the dance
somewhere. Are these small bites I am seeing prior to our deep crawl the effect of google crawling higher pr sites with
inbound links to us, and after our crawl from inbound links of lower pr sites? -correct me if I am wrong.

Also for 4 months running now I have noticed an unknown bot in the logs exactly a day before google shows up. Is this a
coincidence, or has anyone else noticed this? I gave it up as coincidence until this past dance...thought there may be more to
it. Now when I see the unknown bot show, I set out the snacks for google.

Marcia

3:06 am on Sep 6, 2002 (gmt 0)

WebmasterWorld Senior Member marcia is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Welcome to WebmasterWorld, argusdesigns. I've got one that's a low PR5 and it seems to get later than anyone else.

This strange bot, do you happen to have an IP number for it? If so, is it consistently the same?

argusdesigns

3:36 am on Sep 6, 2002 (gmt 0)

10+ Year Member



Hi marcia, thanks for the welcome.
As far as an IP #, no. But have since installed new web logs that track IP's so I will keep you posted on the results come the next dance. I dub the little critter "her shadow".

Thanks for the verify on the pr5.. sounds logical.

savvy1

5:54 am on Sep 6, 2002 (gmt 0)

10+ Year Member



more details about unknown bot? Maybe a rudimentary cloak checker?

rfgdxm1

6:20 am on Sep 6, 2002 (gmt 0)

WebmasterWorld Senior Member rfgdxm1 is a WebmasterWorld Top Contributor of All Time 10+ Year Member



>I've got one that's a low PR5 and it seems to get later than anyone else.

Precisely how do you know it is a low PR5 and not a high PR5?

argusdesigns

12:29 pm on Sep 6, 2002 (gmt 0)

10+ Year Member



I am guessing that a low pr5 toggles between pr4-pr5 during the update. A high pr5 toggles between pr5-pr6 during the
update. I have heard people refer to different echelons of pr, but have not been able to really find any more info. on it.

more details about unknown bot? Maybe a rudimentary cloak checker?
Not sure about that one, we don't use any sort of cloaking or .asp files, just good ol' fashion html. It has occured to me that this may be the old log software, but i doubt it. I'll know more next dance.

ukgimp

12:47 pm on Sep 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Maybe a rudimentary cloak checker?

Hello Arg....

I think Savvy was saying that it might be a check to see if you are using a cloak of some sort. Cloaking relies on the software being able to recognise a particular search engine and depending on who requests the page different versions are presented

EG
SE = optimised content, keyword rich
User = flash version or similar

Engines know what goes on so they look for sites that serve different pages to different IP addresses, so by using new IP's SE can hopefull (they think) tell the difference.

Cheers

rfgdxm1

1:33 pm on Sep 6, 2002 (gmt 0)

WebmasterWorld Senior Member rfgdxm1 is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Right, ukgimp. If Google wants to catch cloaking, send a bot not identifying as Googlebot, and from IPs Google doesn't use, and compare what is sent to the bot to what Googlebot got. This should easily catch cloakers. Googlebot seems to do a LOT of crawling. Google obviously has the resources to have secret bots running looking for cloakers.

savvy1

3:01 pm on Sep 6, 2002 (gmt 0)

10+ Year Member



Yes, exactly what ukgimp said :) (Thanks ukgimp)
I did not mean to imply that you were cloaking or anything about your site in particular. :)

gsmitchell

5:50 pm on Sep 6, 2002 (gmt 0)

10+ Year Member



I have been listed in google for a little while now, but since my initial listing I have not seen an update. I was under the impression that google does a sweep once a month. Am I correct in assuming this? I have a PR of 4 and was wondering if this is the reason my site's not getting updated
 

Featured Threads

Hot Threads This Week

Hot Threads This Month