Where Can A Newbie Find More Info?

I need to learn more...

[jingleheimer]

I've noticed that Searchengineworld.com and webmasterworld.com are great places to go to learn about Search Engine Optimization, but where does a newbie go to learn about Search Engine Spider Identification besides this forum. I've built a great website (in my opinion) and now I've submitted to enough search engines that I'm finally getting hits on my website. However, I've got many questions. For example, my statistics software (using LiveStats 5.0 software from MediaHouse) shows that I am getting about 5000 sessions a week, but only about 20% of those are referred sessions. Is the other 80% coming from Spiders? I've also noticed that a lot of my stats show stuff like "cache-ra04.proxy.aol.com". These are filling up all over the place and I don't know why they're so popular. Can anyone help or point me to the right direction so I can find out more information? Thank-you so much.

Jon

[Froggyman]

Try the Web Robots Database at:

[info.webcrawler.com...]

It's a little outdated but still has a lot of good info and some good links.

[WebRookie]

Here are more spider links to check out:

[edit: url snipped: bt]

Botspot
[botspot.com...]

Spiderkb.com
[spiderkb.com...]

List of resources:
Intelligent Agents WWW sites
[comp.lancs.ac.uk...]

[Brett_Tabke]

snipped the urls since jk has dropped urls many times this week. nuff.

[Eathan]

Hey cool! WebRookie, you are officially the first person to actually recommend my site, and not as an example of just how "Under Construction" a site can be. :)