Not too long ago i started a thread with the question of "What can i do to get my forum spidered?"
Well i think i have found it, and the key... is RSS.
RSS = Really Simple Syndication and basically is a method of formatting summarized versions of your site's content and submitting it to other sites to use as their own. The other sites will list your rss Feed and in turn you get a bunch of inbound links.
So, what i did is turned ON the RSS option in vbulletin. This gave my site a sweet feed that i could submit to different RSS directories. I took the feed URL and went over to pingomatic and started submitting.
Fastforward about 2 months... my site moved up a PR point (pr4 now), but more importantly the site is getting spidered much more thoroughly.
By submitting my feed to a ton of direcotires i have inbound links pointing @ the majority of my threads and this gets the pages indexed QUICK.
G Seems to be taking it slow and steady with the inclusions of my threads, but its working. I can already see a TON in google and its like a 500% difference. Actual traffic from google is up, but the volume is up maybe 20% max.
I find almost no difference with Yahoo. The traffic i get from yahoo is minimal, probably like 30% of what google brings in, and according to my awstats, yahoo doesnt slurp me that often.
OMG MSN loves it lol. After setting up the RSS feeds and submitting them to the directories daily i have seen MSN go buck WILD on my site. It spiders around 80% more gigs than google does and has ranked my in some great spots for keywords that are totally relevant to the content being discussed. I think MSN really likes NEW fresh content and when you have your content and new inbound links showing up on sites that update frequently... AND your site is always growing (like a forum does with each post and threaD) it is very appealing to MSN.
I would love for some others to try this out and tell me how it goes for you. It cant do anything but good and if your forum script already supports RSS then why not? Now go make some $$