Welcome to WebmasterWorld Guest from

Forum Moderators: rogerd

Message Too Old, No Replies

Forum ONLY spiders session id's

Tested this with Swish-e and another site search spider



6:59 am on Mar 21, 2006 (gmt 0)

10+ Year Member

Major problem:

I run a pretty big forum for musicians (invision board software). Lately, my links have dropped out of the Google index. My wiki gets spidered and indexed fine, but my forum doesn't get indexed anymore. I am confident this is a problem on my part, and I will explain why. I think I have figured out the problem, but I don't know how to solve it:

The search engines are only spidering and indexing my session id's. That's it. How do I know? Luckily, through a control. I used Swish-e (a software site search) to spider my forum about 4 months ago. It spidered everything (session id's, regular urls, etc) However, recently, Swish-e will only spider session id's. If I block the session id's from being spidered (in robots.txt), swish-e will NOT spider and index any part of my forum. Note, swish-e will spider the rest of my site fine.

On another site search software (called orca search) the only way it will spider my forum is if I allow cookies. It will spider the rest of my site fine without the need to allow cookies.

Therefore, I conclude Google is behaving in a similar way as the above spiders. Thus, the reason why my forum is now out of the Google index.

Please help, my site is stagnant because of this problem, and I don't know how to fix it. My site has a pagerank 7, and has been online since 1997 with a great reputation. I don't see any reason for Google to all of a sudden do this. For the reasons I mentioned, I believe this is a technical problem on my side, and not Google's fault. I am guessing this is a cookie problem, but I don't know what to do and I have no idea what I did to create this error. Thanks for taking the time to read.


6:21 pm on Mar 22, 2006 (gmt 0)

WebmasterWorld Administrator rogerd is a WebmasterWorld Top Contributor of All Time 10+ Year Member

Does your software have a modification or a setting to suppress session IDs for spiders? Spiders can't normally take cookies, so your forum software may feed them a SessionID. Most forum packages have either build-in or add-on hacks to identify Googlebot et al and not give them the session IDs.

Have you done any upgrades or patches that might have changed or written over the session ID settings/code?

When you say they are indexing ONLY the session IDs, what do the URLs look like?



2:28 am on Mar 23, 2006 (gmt 0)

10+ Year Member

The id's look like this: example.com/index.php?SID=9876546399338645424678

I think my best bet is to re-install invision board from scratch, because I really don't know what I did to have caused this problem. I have some modifications, maybe when I installed a modification, something got messed up.

I am wondering if I should make the switch to VBulletin or not. My search engine rankings are very poor, despite the reputation of the site. Do you know if VBulletin is better suited for larger sites than invision board?


8:51 pm on Mar 23, 2006 (gmt 0)

WebmasterWorld Administrator rogerd is a WebmasterWorld Top Contributor of All Time 10+ Year Member

I can't speak to the comparison. vBulletin is OK, but not spectacular, out of the box. It does have a pseudo-static archive that features very light pages and clean URLs. The forum pages themselves are somewhat bloated and laden with spurious links. It can definitely get traffic, though.

There's also a paid hack that implements multiple SEO fixes.


Featured Threads

Hot Threads This Week

Hot Threads This Month