Welcome to WebmasterWorld Guest from 54.198.179.85

Forum Moderators: open

Message Too Old, No Replies

G Deep Crawling?

     
5:45 am on Oct 1, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member nick_w is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Feb 4, 2002
posts:5044
votes: 0


Noticed that during the night instead of pecking off a few pages as his the daily habbit Gbot picked up a few K's worth on some sites...

Nick

5:08 pm on Oct 1, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:807
votes: 0


I've been geting deepcrawled daily for over a month.
5:12 pm on Oct 1, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 13, 2003
posts:630
votes: 0


I have deep crawls daily ;)
10:32 pm on Oct 1, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:May 14, 2003
posts:376
votes: 0


i don't know that i can tell one from the other... over night, google has been all over my site... last week, it was google, inktomi, scooter, zyborg and a few others... i seem to have more bots visiting than users ;)
10:34 pm on Oct 1, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 20, 2002
posts:56
votes: 0


awstats will report the different bots with decent precision.

if you want to go crazy with precision, get the fantom bot list...

10:50 pm on Oct 1, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 25, 2003
posts:171
votes: 0


Google has been crawling us like crazy since Saturday. *shrug*
5:13 am on Oct 2, 2003 (gmt 0)

Full Member

10+ Year Member

joined:Feb 19, 2003
posts:203
votes: 0


Saw the liitle guy go deep and long today!
6:00 am on Oct 2, 2003 (gmt 0)

New User

10+ Year Member

joined:June 12, 2003
posts:25
votes: 0


Usually 20 pages a day for the GBot, but last night it was double--41 pages.

Dayo_UK

10:53 am on Oct 2, 2003 (gmt 0)

Inactive Member
Account Expired

 
 


Agreed this looks more like a deepcrawl.

100 odd pages taken yesterday, another 80 odd today - and different pages too :)

I am used to having the same 100 pages crawled every few days, but today it is picking up deep new pages.

11:18 am on Oct 2, 2003 (gmt 0)

Senior Member from ZA 

WebmasterWorld Senior Member 10+ Year Member

joined:July 15, 2002
posts:1720
votes: 1


Been deepcrawling for two days now on a few sites.
11:40 am on Oct 2, 2003 (gmt 0)

Senior Member from MT 

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 1, 2003
posts:1843
votes: 0


Hmm between 500 and 6000 pages per day... 2000 on tuesday and hardly anything yesterday... and all of my home computer, ouch. Makes me lag sometimes...

SN

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members