homepage Welcome to WebmasterWorld Guest from 54.234.141.47
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
This month's crawl.
bcc1234

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3937 posted 6:48 am on Jul 1, 2002 (gmt 0)

It this month's crawl over ?
When should the results be reflected ?

Thanks.

 

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3937 posted 8:14 am on Jul 1, 2002 (gmt 0)

They just hit us for 1300 pages in one hour.

ferrari360

10+ Year Member



 
Msg#: 3937 posted 8:16 am on Jul 1, 2002 (gmt 0)

I'm still getting new sites picked up...

bcc1234

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3937 posted 10:56 am on Jul 1, 2002 (gmt 0)

I see, well I lost 900+ pages from one site, so it's not a happy month for me.

The traffic dropped, for a while I thought there was something wrong with the server.

I had a domain mysite1.com and then did 301 redirect to a new domain mysite2.com and then (after google got about 5 pages from mysite2.com) - I did another 301 from mysite2.com to mysite3.com and set mysite1.com to also do a redirect to mysite3.com.

So all pages from mysite1.com dropped, and I have a few pages from mysite2.com and none from mysite3.com.

The site is about 1400 pages, how much time would it take to index under a new domain (right now PR is grey) ?

FreeBee

10+ Year Member



 
Msg#: 3937 posted 12:09 pm on Jul 1, 2002 (gmt 0)

Noticed some deep crawling on the weekend and then looks like one site started getting hammered this morning - maybe it's not over?

MrLucky

10+ Year Member



 
Msg#: 3937 posted 6:17 pm on Jul 1, 2002 (gmt 0)

28th and 29th saw a lot of activity from the bot. Sunday it took a break and all but disappeared. Scared me a little but it's back again today with early signs of a good crawl.

Do we have an official start date? I hoping to see it continue through the 3rd or 4th.

theposter

10+ Year Member



 
Msg#: 3937 posted 6:24 pm on Jul 1, 2002 (gmt 0)

is it true that once he deep crawls you he won't be back ?. aww cmon that must not be true....i hope he comes back again for a deep crawl...he will won't he ??....what do the experts say ?? (everyone here except me is an expert).. ;)

korkus2000

WebmasterWorld Senior Member korkus2000 us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3937 posted 6:31 pm on Jul 1, 2002 (gmt 0)

Googlebot should be back every month to give you a nice deep crawl. If you have a lot of pages each crawl could take several days.

Google finished up on all my domains yesterday. Googlebot usually comes back at the end of a crawl to make sure everything still there.

Big_Balou

10+ Year Member



 
Msg#: 3937 posted 11:43 pm on Jul 1, 2002 (gmt 0)

Googlebot started crawling us on the 28th and is still crawling. Very deep crawl and he's being very nice about it. No hammering here.

Visit Thailand

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3937 posted 11:51 pm on Jul 1, 2002 (gmt 0)

Excuse me for asking but how do you know when exactly the googlebot has come for tea ?

bcc1234

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3937 posted 11:52 pm on Jul 1, 2002 (gmt 0)

Log files, look for Googlebot

olwen

10+ Year Member



 
Msg#: 3937 posted 11:53 pm on Jul 1, 2002 (gmt 0)

Googlebot is still on my site.

I was also over the moon to manage to get a one-page site I did in a hurry spidered on Sunday. I'll add more content later.

mack

WebmasterWorld Administrator mack us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3937 posted 11:56 pm on Jul 1, 2002 (gmt 0)

Googlebot made took just over 2700 get requests in under 6 hours..that aint normal...is it???

Filipe

10+ Year Member



 
Msg#: 3937 posted 12:10 am on Jul 2, 2002 (gmt 0)

Well, after accidentally obliterating 3000 items from my eCommerce site's database, Googlebot continues to spider. They must have crawled 7000 - 9000 pages of my site already (https:// included), and it doesn't look like they're stopping.

podawg

10+ Year Member



 
Msg#: 3937 posted 12:50 am on Jul 2, 2002 (gmt 0)

Googlebot, Missus googlebot and a whole slew of little googlebots are visiting me right now.
Many googlebots.
Is there such a thing as too many googlebots?

dauction

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3937 posted 12:54 am on Jul 2, 2002 (gmt 0)

Filpe..you guys are something else..7000-9000 pages ! and thats just part of your site-s?

man oh man do I have some catching up to do !

I celebrate after I finish 2 pages LOL..

guess iut dosent matter as long as we keep moving forward..

wasmith

10+ Year Member



 
Msg#: 3937 posted 3:28 am on Jul 2, 2002 (gmt 0)

I'm complete :) even the lowest of the low for me (6 hours ago).

Filipe

10+ Year Member



 
Msg#: 3937 posted 4:09 am on Jul 2, 2002 (gmt 0)

Mine looks done, too. dauction - my site has a lot of well-linked to, dynamic pages - only about 200 of them created by hand. The rest are dynamically created when users upload their photos, and Google spiders them all!

Bwahahahaha! (Maniacal laughter 'caused by all his extended url's being crawled and indexed, even with ? and & in them) Thousands I tell you, thousands!

scooch

10+ Year Member



 
Msg#: 3937 posted 11:12 am on Jul 2, 2002 (gmt 0)

GB's still drinking at my well.

2868 pages on Sunday
2152 pages yesterday
1361 page already this morning and still going.

a few thousand to go.

Caught me off-guard. I didn't think it would start as soon as it did. Had to hurry with my updates and I'm afraid I got many in too late. Oh well, I won't miss next month's!

yankee

10+ Year Member



 
Msg#: 3937 posted 7:28 pm on Jul 2, 2002 (gmt 0)

The crawl started on 6/27 and usually lasts 9 days. It should end around 7/5.

ferrari360

10+ Year Member



 
Msg#: 3937 posted 1:00 am on Jul 8, 2002 (gmt 0)

I had my last googlebot visit around 4.00am Saturday morning australian time...

yankee

10+ Year Member



 
Msg#: 3937 posted 1:56 am on Jul 8, 2002 (gmt 0)

It ended here Saturday too. Strange, googlebot came back and re-spidered pages it had already gotten on 6/28. Must be 404 checking or content updating. It wasn't the normal fresh spidering, it was a big blast. Anyone else get a big respider on Saturday?

Dpeper

10+ Year Member



 
Msg#: 3937 posted 3:50 am on Jul 8, 2002 (gmt 0)

One hit me on saturday, too.

Axacta

10+ Year Member



 
Msg#: 3937 posted 3:53 am on Jul 8, 2002 (gmt 0)

Yep - Saturday.

Art

10+ Year Member



 
Msg#: 3937 posted 9:03 am on Jul 8, 2002 (gmt 0)

Hi everybody,
I've got 65 visits from Googlebot on
06 Jul 2002 - 13:16 EST.
I wonder why I do have the new title has been included at wwww.google.com yet only and both the title and the description on www2.google.com? Any ideas?

Art.

fathom

WebmasterWorld Senior Member fathom us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3937 posted 9:18 am on Jul 8, 2002 (gmt 0)

Googlebot still picking up a few in the early morning of July 8.

Will probably have mixed results this crawl. Suffered a hard drive failure (and no redundancy which was part of host's contract) in the middle of googlebot's deep crawl session and was down for 48 hours.

Good old "Murphy's Law" - what can go wrong, will!

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved