homepage Welcome to WebmasterWorld Guest from 54.167.10.244
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
This month's crawl.
bcc1234




msg:58722
 6:48 am on Jul 1, 2002 (gmt 0)

It this month's crawl over ?
When should the results be reflected ?

Thanks.

 

Brett_Tabke




msg:58723
 8:14 am on Jul 1, 2002 (gmt 0)

They just hit us for 1300 pages in one hour.

ferrari360




msg:58724
 8:16 am on Jul 1, 2002 (gmt 0)

I'm still getting new sites picked up...

bcc1234




msg:58725
 10:56 am on Jul 1, 2002 (gmt 0)

I see, well I lost 900+ pages from one site, so it's not a happy month for me.

The traffic dropped, for a while I thought there was something wrong with the server.

I had a domain mysite1.com and then did 301 redirect to a new domain mysite2.com and then (after google got about 5 pages from mysite2.com) - I did another 301 from mysite2.com to mysite3.com and set mysite1.com to also do a redirect to mysite3.com.

So all pages from mysite1.com dropped, and I have a few pages from mysite2.com and none from mysite3.com.

The site is about 1400 pages, how much time would it take to index under a new domain (right now PR is grey) ?

FreeBee




msg:58726
 12:09 pm on Jul 1, 2002 (gmt 0)

Noticed some deep crawling on the weekend and then looks like one site started getting hammered this morning - maybe it's not over?

MrLucky




msg:58727
 6:17 pm on Jul 1, 2002 (gmt 0)

28th and 29th saw a lot of activity from the bot. Sunday it took a break and all but disappeared. Scared me a little but it's back again today with early signs of a good crawl.

Do we have an official start date? I hoping to see it continue through the 3rd or 4th.

theposter




msg:58728
 6:24 pm on Jul 1, 2002 (gmt 0)

is it true that once he deep crawls you he won't be back ?. aww cmon that must not be true....i hope he comes back again for a deep crawl...he will won't he ??....what do the experts say ?? (everyone here except me is an expert).. ;)

korkus2000




msg:58729
 6:31 pm on Jul 1, 2002 (gmt 0)

Googlebot should be back every month to give you a nice deep crawl. If you have a lot of pages each crawl could take several days.

Google finished up on all my domains yesterday. Googlebot usually comes back at the end of a crawl to make sure everything still there.

Big_Balou




msg:58730
 11:43 pm on Jul 1, 2002 (gmt 0)

Googlebot started crawling us on the 28th and is still crawling. Very deep crawl and he's being very nice about it. No hammering here.

Visit Thailand




msg:58731
 11:51 pm on Jul 1, 2002 (gmt 0)

Excuse me for asking but how do you know when exactly the googlebot has come for tea ?

bcc1234




msg:58732
 11:52 pm on Jul 1, 2002 (gmt 0)

Log files, look for Googlebot

olwen




msg:58733
 11:53 pm on Jul 1, 2002 (gmt 0)

Googlebot is still on my site.

I was also over the moon to manage to get a one-page site I did in a hurry spidered on Sunday. I'll add more content later.

mack




msg:58734
 11:56 pm on Jul 1, 2002 (gmt 0)

Googlebot made took just over 2700 get requests in under 6 hours..that aint normal...is it???

Filipe




msg:58735
 12:10 am on Jul 2, 2002 (gmt 0)

Well, after accidentally obliterating 3000 items from my eCommerce site's database, Googlebot continues to spider. They must have crawled 7000 - 9000 pages of my site already (https:// included), and it doesn't look like they're stopping.

podawg




msg:58736
 12:50 am on Jul 2, 2002 (gmt 0)

Googlebot, Missus googlebot and a whole slew of little googlebots are visiting me right now.
Many googlebots.
Is there such a thing as too many googlebots?

dauction




msg:58737
 12:54 am on Jul 2, 2002 (gmt 0)

Filpe..you guys are something else..7000-9000 pages ! and thats just part of your site-s?

man oh man do I have some catching up to do !

I celebrate after I finish 2 pages LOL..

guess iut dosent matter as long as we keep moving forward..

wasmith




msg:58738
 3:28 am on Jul 2, 2002 (gmt 0)

I'm complete :) even the lowest of the low for me (6 hours ago).

Filipe




msg:58739
 4:09 am on Jul 2, 2002 (gmt 0)

Mine looks done, too. dauction - my site has a lot of well-linked to, dynamic pages - only about 200 of them created by hand. The rest are dynamically created when users upload their photos, and Google spiders them all!

Bwahahahaha! (Maniacal laughter 'caused by all his extended url's being crawled and indexed, even with ? and & in them) Thousands I tell you, thousands!

scooch




msg:58740
 11:12 am on Jul 2, 2002 (gmt 0)

GB's still drinking at my well.

2868 pages on Sunday
2152 pages yesterday
1361 page already this morning and still going.

a few thousand to go.

Caught me off-guard. I didn't think it would start as soon as it did. Had to hurry with my updates and I'm afraid I got many in too late. Oh well, I won't miss next month's!

yankee




msg:58741
 7:28 pm on Jul 2, 2002 (gmt 0)

The crawl started on 6/27 and usually lasts 9 days. It should end around 7/5.

ferrari360




msg:58742
 1:00 am on Jul 8, 2002 (gmt 0)

I had my last googlebot visit around 4.00am Saturday morning australian time...

yankee




msg:58743
 1:56 am on Jul 8, 2002 (gmt 0)

It ended here Saturday too. Strange, googlebot came back and re-spidered pages it had already gotten on 6/28. Must be 404 checking or content updating. It wasn't the normal fresh spidering, it was a big blast. Anyone else get a big respider on Saturday?

Dpeper




msg:58744
 3:50 am on Jul 8, 2002 (gmt 0)

One hit me on saturday, too.

Axacta




msg:58745
 3:53 am on Jul 8, 2002 (gmt 0)

Yep - Saturday.

Art




msg:58746
 9:03 am on Jul 8, 2002 (gmt 0)

Hi everybody,
I've got 65 visits from Googlebot on
06 Jul 2002 - 13:16 EST.
I wonder why I do have the new title has been included at wwww.google.com yet only and both the title and the description on www2.google.com? Any ideas?

Art.

fathom




msg:58747
 9:18 am on Jul 8, 2002 (gmt 0)

Googlebot still picking up a few in the early morning of July 8.

Will probably have mixed results this crawl. Suffered a hard drive failure (and no redundancy which was part of host's contract) in the middle of googlebot's deep crawl session and was down for 48 hours.

Good old "Murphy's Law" - what can go wrong, will!

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved