Welcome to WebmasterWorld Guest from 54.162.240.235

Forum Moderators: open

Message Too Old, No Replies

G Deep Crawling?

     
5:45 am on Oct 1, 2003 (gmt 0)

WebmasterWorld Senior Member nick_w is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Noticed that during the night instead of pecking off a few pages as his the daily habbit Gbot picked up a few K's worth on some sites...

Nick

5:08 pm on Oct 1, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I've been geting deepcrawled daily for over a month.
5:12 pm on Oct 1, 2003 (gmt 0)

10+ Year Member



I have deep crawls daily ;)
10:32 pm on Oct 1, 2003 (gmt 0)

10+ Year Member



i don't know that i can tell one from the other... over night, google has been all over my site... last week, it was google, inktomi, scooter, zyborg and a few others... i seem to have more bots visiting than users ;)
10:34 pm on Oct 1, 2003 (gmt 0)

10+ Year Member



awstats will report the different bots with decent precision.

if you want to go crazy with precision, get the fantom bot list...

10:50 pm on Oct 1, 2003 (gmt 0)

10+ Year Member



Google has been crawling us like crazy since Saturday. *shrug*
5:13 am on Oct 2, 2003 (gmt 0)

10+ Year Member



Saw the liitle guy go deep and long today!
6:00 am on Oct 2, 2003 (gmt 0)

10+ Year Member



Usually 20 pages a day for the GBot, but last night it was double--41 pages.
10:53 am on Oct 2, 2003 (gmt 0)

10+ Year Member



Agreed this looks more like a deepcrawl.

100 odd pages taken yesterday, another 80 odd today - and different pages too :)

I am used to having the same 100 pages crawled every few days, but today it is picking up deep new pages.

11:18 am on Oct 2, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Been deepcrawling for two days now on a few sites.
11:40 am on Oct 2, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hmm between 500 and 6000 pages per day... 2000 on tuesday and hardly anything yesterday... and all of my home computer, ouch. Makes me lag sometimes...

SN

 

Featured Threads

Hot Threads This Week

Hot Threads This Month