Is there a way to get Gbot to rapid-crawl a site?

What gets some pages get crawled and indexed, immediately..?

[a_chameleon]

This little tidbit gets me curious.

In another thread, there is a link in GG's post to [google.com ].

In this SERP - there is already a listing for the GoogleGuy's post in the thread I reference. Today is 1/11/04. GG posted his message on on 1/9/04 ands it's already cached in Google :o

Now, I know that once Gbot crawls a page, and decides to index/cache the page, it takes about 24hrs for the page to show in SERP's.

Could somebody enlighten me as to what gets Google to index the page where GG's post is in less than a day?

The page containing GG's post has no PR, and I doubt very seriously if there were (within 24 hrs) a host of backlinks to this page for Gbot to find.

What's the secret for this "instant indexing"? Anyone know?

;)

[taxpod]

Links from "important" pages. "Important" pages are, IMHO, pages linked to from "important" sites. I'm not entirely certain what makes an "important" site but suffice it to say that it has a home page with high PR (WebmasterWorld has a 7 on the front door), it ranks high for "important" keyword searches (whatever that means), it contains rapidly changing content.

So if Gbot is frequently all over a site and that site generally ranks well, new links on that site will get crawled rapidly. The reason for no PR is that PR doesn't get updated quite as rapidly as searches do. But odds are the page in question will have PR soon.

I know of a place where I can get new pages linked to quickly. This place is an "important" page on an "important" site and the new page gets crawled generally within one or two days. Typically the page will be searchable within another day and PR generally gets there within a month or so.

[takagi]

Having a link from a page that is often spidered will do the trick. Googlebot likes pages that have proven to often change in the past (not just a new timestamp, but new content, new links, etc) and have a decent PR. It's not just because of GoogleGuy writing in that thread. I've seen a posting I made at WebmasterWorld show up in the SERPs within 24 hours and GoogleGuy didn't write anything in that thread.

So if you link a new sub page from your home page or from a special "What's new Page" on your site, then that page is more likely to be spidered.

[HarryM]

Similar to what Takagi recommended, I have an area on my home page "Recent Pages" to which I add links to new pages as they are created. As soon as the page shows a PR0, I remove the link. It certainly speeds up indexing.