homepage Welcome to WebmasterWorld Guest from 54.226.173.169
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

This 120 message thread spans 4 pages: < < 120 ( 1 2 3 [4]     
Deepbot is in the house....
I just seen deepbot on my server all
teeceo




msg:47528
 10:30 pm on Apr 15, 2003 (gmt 0)

Its deepbot time!

teeceo.

 

Critter




msg:47618
 2:58 pm on Apr 25, 2003 (gmt 0)

Ok, who's tempted to start a "the update has started" thread? :)

Peter

farside847




msg:47619
 3:32 pm on Apr 25, 2003 (gmt 0)

nothing here, although scooter has been busy...

bielle64




msg:47620
 4:09 pm on Apr 25, 2003 (gmt 0)

Right, Scooter is much more efficient than Google these days. Alatavista strikes back?

sharpie_79




msg:47621
 8:40 am on Apr 26, 2003 (gmt 0)

scooter crawling is too fast...
+ results don't show up in the index..
i'm going to block scooter..

WebWalla




msg:47622
 9:52 am on Apr 26, 2003 (gmt 0)

Got hit by 216.239.46.165 less than 6 hours ago.

qwerty03




msg:47623
 10:05 am on Apr 26, 2003 (gmt 0)

Seeing the 216.* as we speak.

Critter




msg:47624
 11:40 am on Apr 26, 2003 (gmt 0)

Deep's in here too.

I agree with the Scooter comment about not showing up in the index...have the same problem (I think) with Ink. Ink crawled 3,500 odd pages and as far as I can see 13 are in the index--boneheaded.

Now, I may be wrong because it hasn't been a full month or so since the crawl, but we'll see.

Peter

renee




msg:47625
 12:51 pm on Apr 26, 2003 (gmt 0)

Deepie (216.239.46.*) is back after stopping the deep crwal 2 days ago. What is going on? Is google re-crawling the entire web? Maybe the last crawl was no good?

Will this delay the next update?

Critter




msg:47626
 1:06 pm on Apr 26, 2003 (gmt 0)

Deep took a break with me too, and is back in today.

Only doing new pages though, it's not a reset.

Peter

renee




msg:47627
 1:11 pm on Apr 26, 2003 (gmt 0)

Critter: Only doing new pages though, it's not a reset.

The Deepie is crawling all my sites, mostly old pages?!

If this is a new, complete crawl then it will take another 10 days to complete!?
=>
Definitely no April update!
May update is in doubt?

Yidaki




msg:47628
 1:30 pm on Apr 26, 2003 (gmt 0)

>Right, Scooter is much more efficient than Google these days. Alatavista strikes back?

*Superlol* - scooter calls itself a crawler. Ok, yah, they crawl. However, new pages take ages to make it into the av index ... at least i hope they will get listed some day ...

Why do you think it's more efficient than google? To me, efficient means - crawl frequently - index continiously - show fresh results within days or hours. Sorry, that's what google does - afaik, no other engine currently does the same.

My newest site (est. April 2) has been crawled by google after it's been online for 5 hours and was listed after less than 36 hours. THAT is efficient!

whiterabbit




msg:47629
 1:43 pm on Apr 26, 2003 (gmt 0)

216.239.46.99
216.239.46.55
216.239.46.42
216.239.46.41
216.239.46.36
216.239.46.33
216.239.46.222
216.239.46.172
216.239.46.116
216.239.46.102

*and*
64.68.82.66

have all just hit my site

(scooter 3.x did as well, wonder if they fought?)

MaxPower




msg:47630
 2:03 pm on Apr 26, 2003 (gmt 0)

What happens if server resets while pages are being crawled? does it start all over or does it pick up from where it got cut off?

dwilson




msg:47631
 3:38 pm on Apr 26, 2003 (gmt 0)

Welcome to WW, MaxPower.

Google's DB holds the results of the crawled pages & URL's to be crawled. So a server reset should not mess anything up -- as long as you're not relying to much on session variables. Pages heavily driven by session vars are not very friendly to a multi-spider crawl anyway, as far as I understand.

sullen




msg:47632
 4:21 pm on Apr 26, 2003 (gmt 0)

still haven't had a full deep crawl.

Did get a full freshbot crawl a few days ago though (64.ip), and those pages aren't showing in the SERPS as fresh. Is Google messing with my mind?

Critter




msg:47633
 4:44 pm on Apr 26, 2003 (gmt 0)

I got a fresh listing on Google right now, but the fresh tag isn't there. Dunno.

Peter

alxdean




msg:47634
 5:30 pm on Apr 26, 2003 (gmt 0)

anybody ever considered that the clever boys and girls over in the googleplex have figured out a way to start a new crawl cycle while one is still underway? Just a thought. Could be the solution to keeping wihtin a fresh index every 30 days with all those thousands of new pages on the net every second.

markis00




msg:47635
 8:54 pm on Apr 26, 2003 (gmt 0)

Guys,
my site has been being nailed by scooter the last couple of days but hasn't been deep or freshed in nearly two weeks. I disappeared from Google and am wondering, could I have been banned?

teeceo




msg:47636
 10:50 pm on Apr 26, 2003 (gmt 0)

Just seen deepbot again(216.239.46.17 [26/Apr/2003:15:46:35), how long will it crawl? And If it crawls this long, when can we expect a update?

teeceo.

Jesse_Smith




msg:47637
 12:51 am on Apr 27, 2003 (gmt 0)

I last got the deepcrawl seven hours ago, chonping away at a section that I put up four days ago! I'm guessing the May update will start between the 10th and 15th.

markis00




msg:47638
 9:47 am on Apr 27, 2003 (gmt 0)

Can someone please reply to my above post
I am worried

Rick_M




msg:47639
 12:03 pm on Apr 27, 2003 (gmt 0)

I noticed freshbot today picking up dynamic pages with 3 and 4 variables. Looks like something new to me.

DarrylParker




msg:47640
 12:56 pm on Apr 27, 2003 (gmt 0)

My brand spanking new site got deepbotted for the second time this crawl last night, at the same time as a freshie was in there too. Unfortunately my pulled-out hair is still dusting my workstation as they continue to ignore my asp pages with query string variables. And I don't have much left.

Can I perhaps blame my PR NULL <grey bar> for why they won't crawl down into my product pages, or should I start retooling my entire URL framework?

Darryl

Critter




msg:47641
 1:18 pm on Apr 27, 2003 (gmt 0)

Well it looks like Deep's done with me. On the other hand Ink's back in :(

Stupid Inktomi, doing a crawl *again* when they crawled 3,500 pages or so on my site last month and included *13* in the index.

Boneheaded...

Peter

skipfactor




msg:47642
 2:23 pm on Apr 27, 2003 (gmt 0)

>Can I perhaps blame my PR NULL <grey bar> for why they won't crawl down into my product pages

That would be my guess. I have a brand new site just indexed last crawl. The ONLY backlink is the $300 Yahoo listing which gives the site a PR3.

Deepbot grabbed all 100 product pages (*.asp?myid=99) on the first crawl. However, there are only 5 main static pages so maybe it's good to keep Deepbot bored on the static end of a new dynamic site.

bielle64




msg:47643
 7:33 pm on Apr 29, 2003 (gmt 0)

I think the Fully is over now :(
Anyone still have visits ?

I just had a visit on my index :'(

bielle64




msg:47644
 8:06 pm on Apr 29, 2003 (gmt 0)

I mean I ONLY had a visit on my index page.

sullen




msg:47645
 8:23 am on Apr 30, 2003 (gmt 0)

bielle64 - I have a very well established site that also didn't see deepbot on any page bar the index. However, it did get a full fresh-crawl over a week ago and those pages aren't showing as fresh.

I think Google may have changed servers or something.

Also agree that it seems better with dynamic pages - had 200 dynamic pages crawled on a brand new site this crawl.

meinereiner




msg:47646
 7:50 am on May 1, 2003 (gmt 0)

Unfortunately no deepbot or freshbot in the last 3 days now :( But hey, he was there before, next googledance near? :D

Adam_C




msg:47647
 4:07 pm on May 1, 2003 (gmt 0)

Deepbot and Fresh are becoming one....

This 120 message thread spans 4 pages: < < 120 ( 1 2 3 [4]
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved