google spiders, where are they?

[h20color]

I've seen others in this forum mention that they track the frequency of Google spiders. I've only had a visit 2 weeks ago in at least 8 months. My site has been up since 1998 and was ranked #1 for for my keywords for years.
How do I invite the spiders to visit more frequently? Since the last visit my site has dropped down to the high 600's.

Thank you,
MMN
.

[Imaster]

For one of my high ranked (PR) site, Googlebot has been following regularly till date, but for a new site that I launched 20 days back, Googlebot has not even picked up 1% of the pages even though there are a few links pointing to it, including several 7 PR links. The bot visited around 1% of the pages 7 days back, but after that it has been coming and fetching only the index page and no other pages. Anyone else?

[quotations]

A brand new site was visited on the 15th and for the main page only and then on the 16th, the entire site was crawled. Only the main page was crawled on the 17th and there have been no visits today.

An older site has been getting about 50% of the pages crawled daily.

[Imaster]

quotations,

Is the new site large or small?

[g1smd]

What is your definition of large then?

[Imaster]

By large, I meant a site with thousands of pages. By small, I meant an average company/etc site with around 10-50 pages.

[futureX]

I cant get rid of Googlebot, its hit my site around 45,000 times this month, and my site only has around 5000 pages, have I trapped it :/

[slade7]

I've seen 21,000 hits from googlebot this month, and I've moved up in the serps to where I was before whatever tweak that was that they did last march or april. Guessing I have about 30,000 pages total based on Google's cache.. so they haven't got all of them yet I suppose ;)

[ds98127]

hi...site launched less than a month ago... urls are like www.mysite.com/viewtopic.php?p=130#130
since as for now only the bulletin board is up..
google visits everyday...but only one page..does not see aything else ..even a few absolute urls that are well parsed ... people said it takes 4-9 weeks for a deep crawl...on 7 january i had a "deep" crawl when it saw one more page but that's it..since then ..only one page..
I made a sitemap and fed it to be crawled.. waiting for 2-3 days to see the results
cheers
niko

[taxpod]

I've seen Gbot every day this month with an average of 4,200 pages per day pulled down. This is pretty typical for me.

[quotations]

>quotations,
>Is the new site large or small?

It is about 600 pages so far.

It will grow about 500-1000 pages per week for the next couple of months and then should level out until next year around this time.

[andy_boyd]

I launched a new site last week, bought the domain / hosting last Monday 12th January, uploaded all 10 pages on Tuesday and pointed 2 links from other relevant sites I own. Then got a link from a local directory on Wednesday.

Googlebot has visited once per day on 16th, 17th and 18th and I've now been included in their SERP's.

However, on an older site with around 100 pages Googlebot vanished for weeks and returned for a full deep crawl last week. It's behaviour has been very erratic recently.

[div01]

Googlebot was around today...

[h20color]

How do I get the bots to visit me more frequently (see original post up-thread)? Do I add robot meta tags? I removed those after FLA. I'm astonished at the frequency of visits for some of the posters here.
Does site size have anything to do with it? Mine is small and the content doesn't change significantly over time.

[Essex_boy]

The bot appears to come to me once a month, I have meta tags telling it to come weekly - it doesnt. So I dont think thats the problem.

The pages that get Botted - to coin a new phrase, are pages which I update weekly. However it took me at least six months to get listed in Google. So hang on in their. Oh yeah I did this.

Because the bot wasnt crawling the entire site I submitted ALL the pages to the adurl section.

[g1smd]

>> I have meta tags telling it to come weekly - it doesnt <<

Google follows its own schedule, not yours. Delete those tags. They are just wasting your bandwidth.

>> I submitted ALL the pages to the adurl section <<

The Google guidelines expressly state not to do this.

[div01]

How do I get the bots to visit me more frequently (see original post up-thread)? Do I add robot meta tags? I removed those after FLA. I'm astonished at the frequency of visits for some of the posters here.
Does site size have anything to do with it? Mine is small and the content doesn't change significantly over time.

More inbound links to your site and regular updates generally lead to Googlebot camping on your website.

[h20color]

Google now shows just 14 links to my site, down from 36 links prior to FLA. I'm working on restoring links. Why would the other 12 links have been dropped from Google's listings? I've also been trying to update my site, although because of the subject matter it's not relevant to update frequently.
How many links are neccesary?

Thank you,
MMN

[div01]

Google only shows backlinks with PR greater than or equal to 4.

[quotations]

>Google only shows backlinks with PR
>greater than or equal to 4.

Google misses a lot of backlinks which are greater than PR 4.

I have one site which has ove 400 backlinks which are greater than PR 4. Google reports it a zero backlinks.

I have dozens of sites where there are hundreds more PR 4 backlinks than google reports.

This latest backlink update dropped 400 - 500 backlinks from several sites even though they are still PR 5 and PR 6 backlinks.

Only a couple of sites went up this time, one from zero to 13 and one from 128 to 276.

[g1smd]

I have a site where the backlinks to www.domain.com are reported as ZERO, but the backlinks to www.domain.com/ (with a trailing slash) are reported to be about 50, which is actually about right.

[Stefan]

g1smd, that brings to mind a certain post of mine that I later edited out... :-)

There are quite a few variations on that theme. Perhaps some of those whose stickys I didn't answer can ponder that.

[quotations]

Nope.

link:http://www.domain-in-question.com/index.html

returns 0 backlinks

link:http://domain-in-question.com/index.html

returns 0 backlinks

link:http://www.domain-in-question.com/

returns 0 backlinks

link:http://domain-in-question.com/

returns 0 backlinks

link:http://www.domain-in-question.com

returns 0 backlinks

link:http://domain-in-question.com

returns 0 backlinks

link:www.domain-in-question.com/index.html

returns 0 backlinks

link:domain-in-question.com/index.html

returns 0 backlinks

link:www.domain-in-question.com/

returns 0 backlinks

link:domain-in-question.com/

returns 0 backlinks

link:www.domain-in-question.com

returns 0 backlinks

link:domain-in-question.com

returns 0 backlinks

on alltheweb.com 47,308 results reported but only 1010 backlinks are actually shown.

on altavista.com 2,437 results

Any other ideas?

[quotations]

> The results show a sample of the links
>that point to your page,

But they show none of the links that point to my page, not a sample, not a few, not most, ...

NONE, zero, zip, nada.

[lazurus]

Why not contact Google? [e-mail]

[g1smd]

I have found that doing an @domain search, that is an @ symbol followed by the domain name, but without the .com or .co.uk part lists a lot of pages that have links to my sites.

That list is completely different to anything that turns up using any sort of variation of the link:whatever command for that site.

[quotations]

>Why not contact Google? [e-mail]

I wrote to [e-mail] and put googleguy and Florida in the subject and they wrote back several times saying that I was wrong, i.e., that the site is fully indexed in google. After about five or six times back and forth they said they would tell engineering about it, as if it was an aberration.

[lazurus]

the site is fully indexed in google

Is the problem no backlinks showing or that no pages of your site are indexed?

[nakulgoyal]

I just had a new website created with over 500 pages of content, 100% unique and with just one incoming link from a good high PR website, all the 500+ pages are indexed and have started getting me traffic via search. I now see that my content has lot of TOP10 listings with PR0 pages.

I wonder what will happen when all my pages are PR5. :-)

[questwtg]

Me too! I just setup a new software for sale site about a week and a half ago. Today it in the top ten for my two keywords, It has about five links, and I never submitted it to Google. So Google spider is working hard!