Is Google spidering tomorrow?

Google spidering

[cazgh]

According to my logs, the Google Bot comes every 10 days and is due tomorrow.

Anyone agree?

[mipapage]

cazgh,

For some perspective, the bot comes to my site pretty much every day, though I wouldn't call her regular.

[ciml]

If you have access to a server with many sites on, then you get to see how Googlebot' patterns are very different from site to site; page to page.

Sometimes daily, sometimes more like monthly.

[ILLstyle]

I would agree I monitor a few sites. Most get hit every day but a few don't.
It seems the sites that get hit every day do better in serps have better PR and so on.
I think sites that have a higher PR get more attention form the bots.

[rfgdxm1]

My sites typically get hit daily. I'd be alarmed if Googlebot was absent for more than a few days.

[Hobbyist]

Mine too! Am i one of the big boys now? :P

[GranPops]

Getting hit virtually every day, and updated. Now 127 No. 1's on G, wish I was young like you lot and had the energy to do this for a living.

Granpops

[trillianjedi]

I think sites that have a higher PR get more attention form the bots.

Correct.

Also sites that have regular fresh content will get hit more regularly.

TJ

[Imaster]

My sites get hits from Google almost everyday, but since the last 3 days I am noticing a huge number of hits from Googlebot, so that calls for a new update soon :)

[Stujoe]

Googlebot has been gulping up pages from my site for about the last week. I did have a couple of important pages show up a few days ago for the first time but they disappeared yesterday. Second time that has happened in a week.

[bobosse]

Could you guys told me what you think about this?

I rebuilt a site a month and a half ago (completly rebuilt) and it got crawled by Googlebot at the beginning of the month during 6 days in a row. Since then, nothing, no Googlebot.

Well, in the SERPs, I could find the old pages, but since the past 10 days, it came down from 20 pages to 2 pages only available today.

Would that mean that I will get all my new pages in the SERPs soon?

Thanks for the feedback.

Bobosse

[g1smd]

I have a site (online at the end of March) that hasn't been freshed for many many weeks. It is at #1. However, at #2 is a page that points to that site, and that page has been fresh many times, but not recently.

Another site, first online at the beginning of May, and indexed on 2003-06-16 was freshed on every page, every single day from 2003-07-07 to 2003-07-19, but hasn't been freshed since then. [Exception: a newer page was freshed once on 2003-07-21 and just that page].

[xlcus]

I would agree I monitor a few sites. Most get hit every day but a few don't.

Yeah, I run quite a lot of sites and I see quite a variation in the frequency of Googlebot accesses...

I run a script to check the last bot access for each site and order the list by date to see which sites were most recently indexed.

Here are the current results...

Jul 23 21:56 www.site-g.com/robot_log
Jul 23 21:48 www.site-j.com/robot_log
Jul 23 19:22 www.site-b.com/robot_log
Jul 23 19:22 www.site-o.com/robot_log
Jul 23 16:56 www.site-q.com/robot_log
Jul 23 11:37 www.site-a.com/robot_log
Jul 23 06:56 www.site-p.com/robot_log
Jul 23 05:05 www.site-e.com/robot_log
Jul 21 02:42 www.site-m.com/robot_log
Jul 21 01:30 www.site-d.com/robot_log
Jul 20 20:30 www.site-h.com/robot_log
Jul 14 18:47 www.site-r.com/robot_log
Jul 11 15:02 www.site-l.com/robot_log
Jul 11 10:00 www.site-i.com/robot_log
Jul 09 16:13 www.site-c.com/robot_log
Jul 09 00:09 www.site-t.com/robot_log
Jul 05 03:09 www.site-n.com/robot_log
Jul 05 03:05 www.site-s.com/robot_log
Jul 05 03:05 www.site-k.com/robot_log
Jul 04 23:53 www.site-f.com/robot_log

[athinktank]

I would agree I monitor a few sites. Most get hit every day but a few don't

I concur. All over the map for my sites. Most all of the PR is the same on all of my sites so can not comment on the hight PR gets more attention theory.

[mipapage]

Hmmm...

Just a point of interest - as I mentioned above we get hit pretty much every day, but yesterday (the 'tomorrow' referred to in the subject) we got hammered. Checked back in the logs and sure enough there is a rough ten day pattern as to when we get a real thourough visit from g-bot.

[Imaster]

As I mentioned earlier, yes Google has been aggresively crawling the sites since a week or so. Good days are ahead :)

[g1smd]

Missing fresh tags for the past 4 days are back today. Lots of fresh seen.

[JonB]

heavy spidering today from gbot.....

[Dayo_UK]

This thread is worrying me.

I had seen Googlebot everyday but now no visit since 22nd :( - while others are reporting heavier activity I am reporting lighter - oh dear.

[Dayo_UK]

Hi All - Now I know y'all are gonna say this is a coincedence but with reference to my last post above - I was getting desperate for a Googlebot visit and pressed the smiley face on the toolbar - and guess what 10mins later Googlebot started visiting - lol.

[pchristensen]

My site is only about a two months old, but Google was spidering my page every two days or so.

Then, a few days ago, every cached page on the site now dates back to two months ago. What is happening?

-Paul

[Napoleon]

I don't want to hear any of this. We had a disaster yesterday... the site has been offline for 36 hours and has another 18 to go.

I fear the worst. I just don't want to think about it!

Someone tell me you can survive 54 hours downtime!

[bcolflesh]

I hope you have an uptime contract with the host!

[Napoleon]

>> I hope you have an uptime contract with the host! <<

Oh... don't start me! It's been an absolute and total nightmare.

You know the feeling... distress, frustration and anger all rolled into one. Knowing that there is nothing you can personally do about it, and cursing everything that moves. Praying that Googlebot will stay away for a few days, but knowing it won't.

When it is back up tomorrow, the next phase kicks in. Worrying about the implications. Checking Google every 5 minutes for signs of impending disaster.

Someone tell me it will be OK... or you'll see a grown man cry!

[kevinpate]

> Someone tell me you can survive 54 hours downtime

The small nfp site I agreed to service 18 months back, when I was absolutely greener than popeye's spinach, survived near on 3 weeks worth of dead time in April '02.

The prior hosting arrangment went dark sans warning, but they were sending word to the nfp org everything would be straightened out. Being both a newbie webhead and a volunteer, I had little choice but to defer to those who accepted the explanations. I did begin to start hunting alternatives just in case, a wise course of action as it turned out. All was well again in short order.

Excluding a couple of self-inflicted newbie hiccups, the general pattern over the past 15 months has been steadily improving visibility and traffic for our regularly expanding content base.

I'm sure it helped some that for our most common 2 word phrase the typical number of results returned are a tad over 1.4 million. It likely would have played differently in a field where the returns total 2, 4 or 6 fold that level.

[incywincy]

sorry to say this but googlebot absolutely hammered my site today. 2277 pages out of a site total of 3241 pages, you had it right for me cazgh.

[Stujoe]

Seems googlebot has pretty much left my site alone today. Just a handful of hits and they were all requests for pages that have been deleted from my site and google results for a couple months.

[xlcus]

I've only had a few Googlebot accesses today. Even on my higher PR pages. This is the first day in ages that I've seen a lull.

[Napoleon]

Well... the site is back at its new host.... and you guys are telling me that crawling has scaled down! I think I need to get drunk!

[too much information]

Dayo_UK, your trick didn't work for me. :(

I havn't seen GB in a LONG time. Last time she was around, I was editing some text (dynamically generated site) and I think I made her mad, because before that she was around all the time.

Does anyone think that sending flowers might help? ;)