Will google spider these pages?

Forum Moderators: open

Message Too Old, No Replies

Will google spider these pages?

brakkar

6:00 pm on Dec 17, 2003 (gmt 0)

Hello,
Googles often comes to my site, and does a level 1 spidering, which means it will spider all the links contained on my frontpage but will not go any deeper.

This is good because I don’t have to wait the full site spidering to have my new documents spidered. I think this was part of this summer changes in google ways of operating.

However, I will change my site so it will NOT contain on my frontpage the links to the new documents I often add to my site, so their only chance to get spidered will be during the monthly full spidering.

I may have a workaround: what if I add these links to my new documents on another domain where I can display them. Will google spider them as it did (once or twice a week) when it does its level 1 spider even if they are on another domain?

In other words, I wish to know if google, during its twice a week level 1 spidering, follows and index the links that points to another domain.

Thanks in advance,
Koros

Stefan

1:05 am on Dec 18, 2003 (gmt 0)

brakkar, the googlebots are not that predictable. Some sites get crawled more often than others, possibly because of Page Rank. There is no monthly spidering as such.

Your solution might work, but it will definitely help things if you build your PR by getting more quality incoming links, and of course adding regular content and new pages. Eventually, you will find that googlebot will visit all your pages more often.

ADDED: And welcome to WebmasterWorld... sorry, man, just noticed that you joined today.

millie

10:55 am on Dec 18, 2003 (gmt 0)

Welcome Koros ...
and Google recommends sites include a site map so that might help to get the bots going a bit.

Bobby

11:22 am on Dec 18, 2003 (gmt 0)

what if I add these links to my new documents on another domain

Hi Koros,

I think that is an excellent idea, not only because it increases the likelihood of those pages or documents getting spidered, but because it networks your domains together.

If they share a common theme then your little 'family' of sorts will grow in importance and you will have more flexibility than by putting all your marbles in one bag.