Getting Google to crawl my site.

[flyfisher812]

Hello. I am new to Google Adsense and have a quick question regarding the Google crawl and incorrect ads. Our site (<snip>) was crawled by Google...only once and has never been back. I did notice in my website traffic reports that the Googlebot visited yesterday and the day before, but the cached page has not been updated. Any help?

Appreciate any help.

A.J.

[edited by: Jenstar at 3:22 pm (utc) on June 13, 2005]
[edit reason] No URLs / site reviews, as per TOS, thanks! [/edit]

[PumpkinHead]

Hi,

Firstly, I'd remove the link to your page as this breaks the tos for this site.

I'm making a guess here, but I think you may have only been visited once when you submitted your site to the search engines.

From what I can see, you don't have any inbound links to your site from other relevant (cat) sites. This would be my first priority.

I'd also advise submitting your site to more than just Google. Try adding your site to Yahoo and MSN etc too.

Other things you may want to consider: Make your pages valid HTML, Add ALT text to your images,

This is simply my personal advice : Remove the "Change Page" fade effect javascript. It may look nice to you, but it simply has no purpose.

Good luck :)

[flyfisher812]

Thanks for the advice. But why would the Googlebot visit but not crawl/update my site? that is what is curious to me. Do I have anything in my site that is turnig the bot away? Should I use/not use a robots.txt file?

[PumpkinHead]

Hi flyfisher,

Google uses its own algorithm to determine if your site is 'worth' crawling in full. The number of 'relevant' sites linking to your site is part of this algorithm. As you get more links, Google will index more of your site.

Regarding the robots.txt, this is only used to STOP robots crawling your site (e.g spam robots that pick up email addresses from your html).

Also, in my last post I said about removing your 'page fade' java script. The reason I said this is because robots don't like it very much and may stop crawling your site as soon as it picks up any excessive javascript code.

Hope this help, good luck to ya :)