Googlebot.. where art thou?

Forum Moderators: open

Message Too Old, No Replies

Googlebot.. where art thou?

ferrari360

1:58 am on Jul 30, 2002 (gmt 0)

Hi Guys,

just wondering if everyone has seen googlebot for this months crawl.. I'm starting to get a little anxious that I've been missed out... had about 4 hours downtime a week or so ago... :(

mack

2:00 am on Jul 30, 2002 (gmt 0)

Just checked logs. Havent had her here either.

Abrexa_UK

10:23 am on Jul 30, 2002 (gmt 0)

Nothing from Googlebot on about half our sites, and much less than we usually get on the others so far :(

chowcat

10:36 am on Jul 30, 2002 (gmt 0)

Nothing for 3 weeks, although the latest update did see some changes for my site!

eplus

10:40 am on Jul 30, 2002 (gmt 0)

I think google has pretty much ignored the UK or my little corner of it at least. My clients have several .coms all hosted in the UK all with page ranks around 5ish and not one has been visited?? It's particularily annoying for me as I have just set up 7 more sites this month and was hoping to get them crawled but no luck. If this is one of googles 'new features' they rolled out with this crawl they can keep it I'd rather have the old googlebot back :-(. How can UK based sites compete with american sites if we don't get indexed properly?
sorry I should have put that in [rant] tags but it's very frustrating to get up everymorning, go to work and check the logs to see no sign of googlebot. Google traffic is very important to many buisnesses and it's things like this that make for bad sales months.
anyway got to go

KakenBetaal

10:47 am on Jul 30, 2002 (gmt 0)

Aaarrrrrggggghhh! I moved hosts via a DNS change, and Googlebot has just hit my index page at the old host. Only one visit, though, not the main crawl I've been hoping for.

Lucky me that I still have all the content as of Friday at the old host. Just imagine I didn't - Google would probably drop me like a hot potatoe.

OK, I've got nothing to complain about as I didn't exactly time the move very well, but I had to do it at this time for other reasons.

[edited by: KakenBetaal at 1:07 pm (utc) on July 30, 2002]

Markus

11:03 am on Jul 30, 2002 (gmt 0)

I've seen the first hits from Googlebot yesterday and within the past few hours she deepcrawled us on several sites. (She also fetched new pages we've uploaded yesterday.)

ukgimp

11:53 am on Jul 30, 2002 (gmt 0)

She is definitly out and about. She is hungry too if my logs of today and the last week are anything to go by.

ferrari360

3:58 am on Jul 31, 2002 (gmt 0)

Hmm.. I still haven't seen googlebot yet... should be soon I imagine.. hows everyone else going in regards to crawling?

badmonkey

4:15 am on Jul 31, 2002 (gmt 0)

Looks like she is back at least temporarily for me. Some crawling yesterday and nothing today until a couple of hours ago.
But here is something interesting I am noticing.
Today the Googlebot is coming from the 216.239.46.0 class C, yesterday it was coming from 64.68.82.0 class C range.

I know that Google recently implemented RouteScience's PathControl optimized routing device.
[infoworld.com...]

Basically it does BGP with added latency intelligence. I don't know if the two are related, but today's bot's ip address is 11 hops off my network vs. yesterday's bot's 23 hops.

What ip addresses are you guys seeing the Google bot come from?

Ciao

podawg

4:24 am on Jul 31, 2002 (gmt 0)

SHE HERE!
and she brought 22 of her little friends! yippee

rfgdxm1

4:25 am on Jul 31, 2002 (gmt 0)

After dropping my entire site in the bit bucket in the latest index, and treating it like it didn't exist any more for 2 weeks (Googlebot usually came for a nip every other day), I just checked my logs and Googlebot is grabbing my entire site. I hope this is a good sign for the future, and this isn't being done just to raise false hopes.

The crawl on my site is coming from the
216.239.46.0 class C block.

podawg

5:14 am on Jul 31, 2002 (gmt 0)

Now she's gone :(
She(they) was only there for about 11 minutes and missed a chunck of my site.
sigh....
off to the store.
maybe it's not too late to buy her some cookies.

shelleycat

6:33 am on Jul 31, 2002 (gmt 0)

After ignoring me for the last couple of weeks googlebot came back today :-D While she didn't deep crawl the entire site she seems to have found all the new pages since last time (I have a blog so there are new pages nearly every day). Strangly blogdex's bot hit me so so many times today also and I don't even think I'm listed with them.

*looks closer* Oh actually, in amogst all those blogdex hits, googlebot did deep crawl every single file on my site. Cool!

KakenBetaal

7:25 am on Jul 31, 2002 (gmt 0)

Oh yeah, Googlebot has found my new web host! Still only hit the index and robots, but it's an improvement on yesterday.

Update: Just looked at my logs on the old host and I'm still seeing the odd visitor coming in there. Jeepers, some ISPs take ages to sync up their DNS servers - it's been 5 days already.

oLeon

5:11 pm on Jul 31, 2002 (gmt 0)

No doubt about it:
she is on the way �round the net...

she got a few thousands of pages today from us, I assume it�s the beginning of the normal spider-cycle.

ferrari360

1:52 am on Aug 1, 2002 (gmt 0)

yep.. I'm starting to see googlebot on a few of my sites now...

jimmykav

7:51 am on Aug 1, 2002 (gmt 0)

Last night I had a massive feeding frenzy from googlebot. It took highest number of pages ever in one session.

ferrari360

7:57 am on Aug 1, 2002 (gmt 0)

It will be interesting to see what happens with this new routescience deal.. google are looking to increase the frequency of their crawling/updates so we may see faster more aggressive crawls in the future - and maybe 2 updates a month... just to put the nail in the coffin of all those who freak out during the monthly update :)