| 2:58 pm on Apr 25, 2003 (gmt 0)|
Ok, who's tempted to start a "the update has started" thread? :)
| 3:32 pm on Apr 25, 2003 (gmt 0)|
nothing here, although scooter has been busy...
| 4:09 pm on Apr 25, 2003 (gmt 0)|
Right, Scooter is much more efficient than Google these days. Alatavista strikes back?
| 8:40 am on Apr 26, 2003 (gmt 0)|
scooter crawling is too fast...
+ results don't show up in the index..
i'm going to block scooter..
| 9:52 am on Apr 26, 2003 (gmt 0)|
Got hit by 220.127.116.11 less than 6 hours ago.
| 10:05 am on Apr 26, 2003 (gmt 0)|
Seeing the 216.* as we speak.
| 11:40 am on Apr 26, 2003 (gmt 0)|
Deep's in here too.
I agree with the Scooter comment about not showing up in the index...have the same problem (I think) with Ink. Ink crawled 3,500 odd pages and as far as I can see 13 are in the index--boneheaded.
Now, I may be wrong because it hasn't been a full month or so since the crawl, but we'll see.
| 12:51 pm on Apr 26, 2003 (gmt 0)|
Deepie (216.239.46.*) is back after stopping the deep crwal 2 days ago. What is going on? Is google re-crawling the entire web? Maybe the last crawl was no good?
Will this delay the next update?
| 1:06 pm on Apr 26, 2003 (gmt 0)|
Deep took a break with me too, and is back in today.
Only doing new pages though, it's not a reset.
| 1:11 pm on Apr 26, 2003 (gmt 0)|
Critter: Only doing new pages though, it's not a reset.
The Deepie is crawling all my sites, mostly old pages?!
If this is a new, complete crawl then it will take another 10 days to complete!?
Definitely no April update!
May update is in doubt?
| 1:30 pm on Apr 26, 2003 (gmt 0)|
>Right, Scooter is much more efficient than Google these days. Alatavista strikes back?
*Superlol* - scooter calls itself a crawler. Ok, yah, they crawl. However, new pages take ages to make it into the av index ... at least i hope they will get listed some day ...
Why do you think it's more efficient than google? To me, efficient means - crawl frequently - index continiously - show fresh results within days or hours. Sorry, that's what google does - afaik, no other engine currently does the same.
My newest site (est. April 2) has been crawled by google after it's been online for 5 hours and was listed after less than 36 hours. THAT is efficient!
| 1:43 pm on Apr 26, 2003 (gmt 0)|
have all just hit my site
(scooter 3.x did as well, wonder if they fought?)
| 2:03 pm on Apr 26, 2003 (gmt 0)|
What happens if server resets while pages are being crawled? does it start all over or does it pick up from where it got cut off?
| 3:38 pm on Apr 26, 2003 (gmt 0)|
Welcome to WW, MaxPower.
Google's DB holds the results of the crawled pages & URL's to be crawled. So a server reset should not mess anything up -- as long as you're not relying to much on session variables. Pages heavily driven by session vars are not very friendly to a multi-spider crawl anyway, as far as I understand.
| 4:21 pm on Apr 26, 2003 (gmt 0)|
still haven't had a full deep crawl.
Did get a full freshbot crawl a few days ago though (64.ip), and those pages aren't showing in the SERPS as fresh. Is Google messing with my mind?
| 4:44 pm on Apr 26, 2003 (gmt 0)|
I got a fresh listing on Google right now, but the fresh tag isn't there. Dunno.
| 5:30 pm on Apr 26, 2003 (gmt 0)|
anybody ever considered that the clever boys and girls over in the googleplex have figured out a way to start a new crawl cycle while one is still underway? Just a thought. Could be the solution to keeping wihtin a fresh index every 30 days with all those thousands of new pages on the net every second.
| 8:54 pm on Apr 26, 2003 (gmt 0)|
my site has been being nailed by scooter the last couple of days but hasn't been deep or freshed in nearly two weeks. I disappeared from Google and am wondering, could I have been banned?
| 10:50 pm on Apr 26, 2003 (gmt 0)|
Just seen deepbot again(18.104.22.168 [26/Apr/2003:15:46:35), how long will it crawl? And If it crawls this long, when can we expect a update?
| 12:51 am on Apr 27, 2003 (gmt 0)|
I last got the deepcrawl seven hours ago, chonping away at a section that I put up four days ago! I'm guessing the May update will start between the 10th and 15th.
| 9:47 am on Apr 27, 2003 (gmt 0)|
Can someone please reply to my above post
I am worried
| 12:03 pm on Apr 27, 2003 (gmt 0)|
I noticed freshbot today picking up dynamic pages with 3 and 4 variables. Looks like something new to me.
| 12:56 pm on Apr 27, 2003 (gmt 0)|
My brand spanking new site got deepbotted for the second time this crawl last night, at the same time as a freshie was in there too. Unfortunately my pulled-out hair is still dusting my workstation as they continue to ignore my asp pages with query string variables. And I don't have much left.
Can I perhaps blame my PR NULL <grey bar> for why they won't crawl down into my product pages, or should I start retooling my entire URL framework?
| 1:18 pm on Apr 27, 2003 (gmt 0)|
Well it looks like Deep's done with me. On the other hand Ink's back in :(
Stupid Inktomi, doing a crawl *again* when they crawled 3,500 pages or so on my site last month and included *13* in the index.
| 2:23 pm on Apr 27, 2003 (gmt 0)|
>Can I perhaps blame my PR NULL <grey bar> for why they won't crawl down into my product pages
That would be my guess. I have a brand new site just indexed last crawl. The ONLY backlink is the $300 Yahoo listing which gives the site a PR3.
Deepbot grabbed all 100 product pages (*.asp?myid=99) on the first crawl. However, there are only 5 main static pages so maybe it's good to keep Deepbot bored on the static end of a new dynamic site.
| 7:33 pm on Apr 29, 2003 (gmt 0)|
I think the Fully is over now :(
Anyone still have visits ?
I just had a visit on my index :'(
| 8:06 pm on Apr 29, 2003 (gmt 0)|
I mean I ONLY had a visit on my index page.
| 8:23 am on Apr 30, 2003 (gmt 0)|
bielle64 - I have a very well established site that also didn't see deepbot on any page bar the index. However, it did get a full fresh-crawl over a week ago and those pages aren't showing as fresh.
I think Google may have changed servers or something.
Also agree that it seems better with dynamic pages - had 200 dynamic pages crawled on a brand new site this crawl.
| 7:50 am on May 1, 2003 (gmt 0)|
Unfortunately no deepbot or freshbot in the last 3 days now :( But hey, he was there before, next googledance near? :D
| 4:07 pm on May 1, 2003 (gmt 0)|
Deepbot and Fresh are becoming one....
| This 120 message thread spans 4 pages: < < 120 ( 1 2 3  ) |