Welcome to WebmasterWorld Guest from 54.160.254.203

Forum Moderators: open

Message Too Old, No Replies

This month's crawl.

     
6:48 am on Jul 1, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 1, 2002
posts:1421
votes: 0


It this month's crawl over ?
When should the results be reflected ?

Thanks.

8:14 am on July 1, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38048
votes: 12


They just hit us for 1300 pages in one hour.
8:16 am on July 1, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:May 3, 2002
posts:89
votes: 0


I'm still getting new sites picked up...
10:56 am on July 1, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 1, 2002
posts:1421
votes: 0


I see, well I lost 900+ pages from one site, so it's not a happy month for me.

The traffic dropped, for a while I thought there was something wrong with the server.

I had a domain mysite1.com and then did 301 redirect to a new domain mysite2.com and then (after google got about 5 pages from mysite2.com) - I did another 301 from mysite2.com to mysite3.com and set mysite1.com to also do a redirect to mysite3.com.

So all pages from mysite1.com dropped, and I have a few pages from mysite2.com and none from mysite3.com.

The site is about 1400 pages, how much time would it take to index under a new domain (right now PR is grey) ?

12:09 pm on July 1, 2002 (gmt 0)

Full Member

10+ Year Member

joined:Feb 19, 2001
posts:308
votes: 0


Noticed some deep crawling on the weekend and then looks like one site started getting hammered this morning - maybe it's not over?
6:17 pm on July 1, 2002 (gmt 0)

New User

10+ Year Member

joined:June 3, 2002
posts:13
votes: 0


28th and 29th saw a lot of activity from the bot. Sunday it took a break and all but disappeared. Scared me a little but it's back again today with early signs of a good crawl.

Do we have an official start date? I hoping to see it continue through the 3rd or 4th.

6:24 pm on July 1, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:June 26, 2002
posts:138
votes: 0


is it true that once he deep crawls you he won't be back ?. aww cmon that must not be true....i hope he comes back again for a deep crawl...he will won't he ??....what do the experts say ?? (everyone here except me is an expert).. ;)
6:31 pm on July 1, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member korkus2000 is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 20, 2002
posts:3732
votes: 0


Googlebot should be back every month to give you a nice deep crawl. If you have a lot of pages each crawl could take several days.

Google finished up on all my domains yesterday. Googlebot usually comes back at the end of a crawl to make sure everything still there.

11:43 pm on July 1, 2002 (gmt 0)

Full Member

10+ Year Member

joined:May 23, 2002
posts:232
votes: 0


Googlebot started crawling us on the 28th and is still crawling. Very deep crawl and he's being very nice about it. No hammering here.
11:51 pm on July 1, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:May 29, 2002
posts:1819
votes: 0


Excuse me for asking but how do you know when exactly the googlebot has come for tea ?
11:52 pm on July 1, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 1, 2002
posts:1421
votes: 0


Log files, look for Googlebot
11:53 pm on July 1, 2002 (gmt 0)

Full Member

10+ Year Member

joined:Mar 18, 2002
posts:309
votes: 0


Googlebot is still on my site.

I was also over the moon to manage to get a one-page site I did in a hurry spidered on Sunday. I'll add more content later.

11:56 pm on July 1, 2002 (gmt 0)

Moderator from GB 

WebmasterWorld Administrator mack is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 15, 2001
posts:7557
votes: 3


Googlebot made took just over 2700 get requests in under 6 hours..that aint normal...is it???
12:10 am on July 2, 2002 (gmt 0)

Preferred Member

10+ Year Member

joined:Apr 25, 2002
posts:470
votes: 0


Well, after accidentally obliterating 3000 items from my eCommerce site's database, Googlebot continues to spider. They must have crawled 7000 - 9000 pages of my site already (https:// included), and it doesn't look like they're stopping.
12:50 am on July 2, 2002 (gmt 0)

New User

10+ Year Member

joined:May 9, 2002
posts:38
votes: 0


Googlebot, Missus googlebot and a whole slew of little googlebots are visiting me right now.
Many googlebots.
Is there such a thing as too many googlebots?
12:54 am on July 2, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 7, 2002
posts:906
votes: 0


Filpe..you guys are something else..7000-9000 pages ! and thats just part of your site-s?

man oh man do I have some catching up to do !

I celebrate after I finish 2 pages LOL..

guess iut dosent matter as long as we keep moving forward..

3:28 am on July 2, 2002 (gmt 0)

Full Member

10+ Year Member

joined:Aug 26, 2001
posts:277
votes: 0


I'm complete :) even the lowest of the low for me (6 hours ago).
4:09 am on July 2, 2002 (gmt 0)

Preferred Member

10+ Year Member

joined:Apr 25, 2002
posts:470
votes: 0


Mine looks done, too. dauction - my site has a lot of well-linked to, dynamic pages - only about 200 of them created by hand. The rest are dynamically created when users upload their photos, and Google spiders them all!

Bwahahahaha! (Maniacal laughter 'caused by all his extended url's being crawled and indexed, even with ? and & in them) Thousands I tell you, thousands!

11:12 am on July 2, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:May 26, 2002
posts:44
votes: 0


GB's still drinking at my well.

2868 pages on Sunday
2152 pages yesterday
1361 page already this morning and still going.

a few thousand to go.

Caught me off-guard. I didn't think it would start as soon as it did. Had to hurry with my updates and I'm afraid I got many in too late. Oh well, I won't miss next month's!

7:28 pm on July 2, 2002 (gmt 0)

Full Member

10+ Year Member

joined:Dec 5, 2001
posts:279
votes: 0


The crawl started on 6/27 and usually lasts 9 days. It should end around 7/5.
1:00 am on July 8, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:May 3, 2002
posts:89
votes: 0


I had my last googlebot visit around 4.00am Saturday morning australian time...
1:56 am on July 8, 2002 (gmt 0)

Full Member

10+ Year Member

joined:Dec 5, 2001
posts:279
votes: 0


It ended here Saturday too. Strange, googlebot came back and re-spidered pages it had already gotten on 6/28. Must be 404 checking or content updating. It wasn't the normal fresh spidering, it was a big blast. Anyone else get a big respider on Saturday?
3:50 am on July 8, 2002 (gmt 0)

Preferred Member

10+ Year Member

joined:May 21, 2002
posts:487
votes: 0


One hit me on saturday, too.
3:53 am on July 8, 2002 (gmt 0)

Preferred Member

10+ Year Member

joined:Apr 15, 2002
posts:398
votes: 0


Yep - Saturday.

Art

9:03 am on July 8, 2002 (gmt 0)

New User

10+ Year Member

joined:Apr 11, 2002
posts:15
votes: 0


Hi everybody,
I've got 65 visits from Googlebot on
06 Jul 2002 - 13:16 EST.
I wonder why I do have the new title has been included at wwww.google.com yet only and both the title and the description on www2.google.com? Any ideas?

Art.

9:18 am on July 8, 2002 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member fathom is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 5, 2002
posts:4110
votes: 109


Googlebot still picking up a few in the early morning of July 8.

Will probably have mixed results this crawl. Suffered a hard drive failure (and no redundancy which was part of host's contract) in the middle of googlebot's deep crawl session and was down for 48 hours.

Good old "Murphy's Law" - what can go wrong, will!