How to get spidered?

I have done everything to make it easy... what now?

[zigx]

Hello again guys. I have done everything in my power to make things easy for SEs. My entire forum is mod_rewritten, all urls are static.

My forum URLs look like this: domain.com/title-of-forum.html

and thread titles look like domain.com/thread1234.html

When i search google i see that barely any pages have been spidered, and some are from the OLD, OLD forum which happens to be like 4 months out of date.

What should i do to get the spiders eating me up? When i look for inbound links should i have people link directly to forums, or to the domain itself?

any input would be great, thanks a lot all.

[Marketing Guy]

The destination of the inbound links doesn't really matter as long as SE spiders can reach your forum from that page. Usually, inbound links point to your homepage and obviously it is beneficial to have links from there to your forums! ;)

It may take some time for your site to get fully spidered and for the pages to be listed in Google.

For example, if you don't have many inbound links, it could just be that spiders haven't found you yet in their regular crawl.

When getting a new site indexed it's advantageous to get inbound links from sites that are indexed fairly regularly. Try registering on a few forums and including your link in your signature (of course, don't spam and only include a sig if the forum allows it! :)).

As long as you have plenty of inbound links from different sources and your site has a spider friendly link structure (ie not too many links on each page, and each page can be reached within a few clicks) then it's just a waiting game - there's not a lot more you can do.

Scott

[rogerd]

I'd recommend that you keep working on the inbound links to promote spidering. Most people will link to your site's home page, but see if you can get some links to your forum home page or even a category page or two. The amount and quality of linkage have a major bearing on spidering frequency and depth.

[zigx]

Since i posted this message ive checked google and have noticed that a TON more pages are now showing up in their SERPs.

I think you guys are right that i may just simply not have many inbound links and this causes the spiders to not come by. I am going to work on linking the forum from various other sites.

Additionally I bought out another forum similarly, and now have it doing 404 redirects (via .htaccess) to my board... Could this be BAD in any way?

[mzadorian]

What forum portal are you using?

[rogerd]

It seems like you'd want 301 redirects to pass any PageRank from the old forum's links.

[zigx]

mzadorian, i am using vbulletin without any type of portal hacks.

[zigx]

hmm.... i will try a 301 redirect and see what happens.