Welcome to WebmasterWorld Guest from 54.226.183.49

Forum Moderators: open

Message Too Old, No Replies

Googlebot is back!

It looks like the deep crawl is starting.

     

RoadRash

9:28 am on Mar 9, 2003 (gmt 0)

10+ Year Member



Google started to deep crawl my site in the last 15 minutes. Anyone else notice this?

ga_ga

9:35 am on Mar 9, 2003 (gmt 0)

10+ Year Member



Yes I'm being invaded too. It's the freshbot here though, on 2 sites. Very hungry, must have had nothing to eat for a few days..

Crawler 11 on 64.68.82.xx

percentages

9:43 am on Mar 9, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I haven't seen the deep crawl bot yet, but to start crawling this fast (within 60 hours of the last update) is a complete nonsense.

Altavista....refreshed within 48 hours, Google.....Refreshed within the last 6 weeks if you are lucky!

Let's start a thread about the next update...sometime in April if Google can recalculate its PR that early.

Google if you can't calculate PR faster than every 6 weeks, stop putting so much importance on something that is way out of date.

EliteWeb

10:20 am on Mar 9, 2003 (gmt 0)

WebmasterWorld Senior Member eliteweb is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I was just thinking what is being talked about here, I haven't seen googlebot to my site in a few days then I look at my network connectivity logs and see tons of googlebots and netscape looking bots connected to it non-stop so guess what, he's bacckkkkkkkkkkkkkkkkkkk.

adsoft13

10:55 am on Mar 9, 2003 (gmt 0)

10+ Year Member



At my place I still saw old results on www 2 hours ago ...
So the update is probably not finished yet :)
But I don't see googlebot an ony of my sites ...

NickH

11:46 am on Mar 9, 2003 (gmt 0)

10+ Year Member



Freshbot was at my site on Friday, took a break on Saturday, and is back today. No sign of the deepcrawler yet.

netguy

11:51 am on Mar 9, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member


Googlebot/2.1 ( http://www.googlebot.com/bot.html) stopped by this morning. No breakdown of whether its 216* or 64*... Is this FreshBot?

steveb

1:35 pm on Mar 9, 2003 (gmt 0)

WebmasterWorld Senior Member steveb is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I've been crawled every day for weeks. The old truism that Freshbot disappears during the update is false.

AthlonInside

2:08 pm on Mar 9, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



> Very hungry, must have had nothing to eat for a few days..

LOL! same here! :)

europeforvisitors

3:22 pm on Mar 9, 2003 (gmt 0)



Google if you can't calculate PR faster than every 6 weeks, stop putting so much importance on something that is way out of date.

PR is only one of many factors that Google uses in determining search placement, so there's no need for it to be recalculated in real time. Don't confuse the desires of Webmasters with the needs of users. :-)

rfgdxm1

3:56 pm on Mar 9, 2003 (gmt 0)

WebmasterWorld Senior Member rfgdxm1 is a WebmasterWorld Top Contributor of All Time 10+ Year Member



>PR is only one of many factors that Google uses in determining search placement, so there's no need for it to be recalculated in real time. Don't confuse the desires of Webmasters with the needs of users. :-)

Right. For the average searcher, if Google is using a 6 week old database this isn't a real problem.

netguy

4:02 pm on Mar 9, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I'm still not completely clear on how much influence FreshBot has with Deep Crawl when it comes to generating the index every month - But I do know that Freshbot is in there almost every day updating our news sites and making it available to the world Looong before the index is completed.

Buckley

5:10 pm on Mar 9, 2003 (gmt 0)

10+ Year Member



Net Guy or anyone, does that mean if you get visits from freshbot, and you made changes to your title tag, you wouldn't have to wait for the next update to see the changes?

Can your position in the serps move based on this or do you have to wait for the dance to see any movement.

Thanks.

joeuz

5:25 pm on Mar 9, 2003 (gmt 0)

10+ Year Member



Buckley, I have experienced this recently. I was changing a page and noticed that the changes were reflected in Google twice in 1 week.

RoadRash

10:29 pm on Mar 9, 2003 (gmt 0)

10+ Year Member



Must have been the freshbot... I see ~50 or so new pages in the index today, just hrs after the crawl. I am expecting some 20,000 dynamic pages to be indexed in the next deep crawl. I have my fingers crossed.

PFOnline

10:35 pm on Mar 9, 2003 (gmt 0)

10+ Year Member



Hi, sorry for newbie question! But, how exactly do we see if Googlebot is at our site?

Jesse_Smith

6:02 pm on Mar 10, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



freshBot: 64.68.82.* bah, listed for only a few days, short dinner date then it dumps you.
deepcrawler: 216.239.46.* Good bot, it likes you, listed until death do you depart. So don't make him mad or it will divorce your site. You can divorce the Googlebot by using your robot.txt file. It's much cheaper and faster than going to the courts.

jdMorgan

6:38 pm on Mar 10, 2003 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Buckley,

Yes to both questions. Our fearless WebmasterWorld leader calls this "surfing the fresh wave."

Jim

peterdaly

7:03 pm on Mar 10, 2003 (gmt 0)

10+ Year Member



Over 3000 hits from GoogleBot so far today.

They are all Freshbot. No need to get alarmed yet about the deep crawl. Not based on information from my server at least.

-Pete

bobmark

7:17 pm on Mar 10, 2003 (gmt 0)

10+ Year Member



I'm wondering if there is a change underway.
I don't know if it is just a change in how Google handles my site or something more.
In the past, freshbot would take my index and main directory pages but nothing else. The past 2 days it is deep crawling the entire site.
Is this a pattern others have experienced in previous months or is it possible that 64.bot is doing 239.bot's work this month?
Ideas?

oLeon

8:44 pm on Mar 10, 2003 (gmt 0)

10+ Year Member



now and then the freshbot crawled very deep on our sites.
some other days it only has a look at the homepage, or some more.

in this moment the freshbot is on his way around, I cannot see the deepbot at all.

4serendipity

3:59 am on Mar 11, 2003 (gmt 0)

10+ Year Member



I got hit by freshbot on the 8th and 9th.

Deepbot started showing around 6 EST today.

WebMistress

4:13 am on Mar 11, 2003 (gmt 0)

10+ Year Member



Google is deep crawling my PR6 site right now...*216...didn't wait long after the update, eh? and I'm not ready for him..of course, I never am no matter how much I optimize all month....I always need mre time...yikes!

joker197cinque

11:08 am on Mar 11, 2003 (gmt 0)

10+ Year Member



Yes....my servers noticed DeepBot 216* 2 times(one at 1 am and one at 4 am italian time )

not ready at all.. :\

bobmark

3:50 pm on Mar 11, 2003 (gmt 0)

10+ Year Member



yeah, I see deepbot now too. Guess I was wrong.
Am I the only one who thinks that, given how late this update was and how late the crawl is we might not see the "March" Update (assuming we define the current update as the "February Update" as the previous was Januiary) until mid to late April? In other words maybe this is gonna be the rare 6-7 weeks between updates that happens once a year or so.
Otherwise, Google will have to be re-indexing on crawl results from March 10 to March 15(?) within 2 weeks or so (yeah, I know they have done that too occasionally - I think August was like that if I recall right).

bluecorr

4:55 pm on Mar 11, 2003 (gmt 0)

10+ Year Member



I just changed some pages and put them up. Googlebot deepcrawled some of the pages on the site prior to my latest update and will obviously come back to take the rest. Will it notice that the pages it had cralwed before are now fresher and if it does will it crawl them again?

Thanks

bether2

4:56 pm on Mar 11, 2003 (gmt 0)

10+ Year Member



Freshbot is busy on my site (PR6). Deepbot (216.239.*) has been "knocking" - ie, hit about 15 pages last night and this morning. That usually means it will start its full-scale deep crawl sometime today or tonight.

Deepbot is usually very polite and knocks first. :) Gives me a little extra time to get those last-minute fixes done.

GoogleGuy

5:42 pm on Mar 11, 2003 (gmt 0)

WebmasterWorld Senior Member googleguy is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Hope everyone has been getting any new content up. :)

PaulPaul

5:52 pm on Mar 11, 2003 (gmt 0)

10+ Year Member



Not at my site yet. :(

bobmark

6:25 pm on Mar 11, 2003 (gmt 0)

10+ Year Member



I believe its been said on other threads but I wanted to say how much I appreciated both the update and deepcrawl headsup, GoogleGuy. Its nice to retain that feeling that we're all in this together.
For myself, I've been working on the assumption that it may be longer than average before the next update; if not, I'll be pleasantly surprised, if so I'll know I did everything possible to get my appropriate season pages in.
This 102 message thread spans 4 pages: 102
 

Featured Threads

Hot Threads This Week

Hot Threads This Month