Welcome to WebmasterWorld Guest from 54.159.246.164

Forum Moderators: lawman

Message Too Old, No Replies

Google & Webmasterworld.com

Why doesn't google index all WebmasterWorld.com pages?

   
2:21 pm on Aug 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hey everybody,

Am I right in assuming that Google does not index all webmasterworld.com forum pages? If yes, why? I see that there are 1000s of plain html files/topics within WebmasterWorld.com but none of them are listed in Google.

If I want to implement a forum in my website, which is the best discussion board software, from Google's "point of view"? YaBB (yet another bulletin board - YabbForum.com)? PHPBB (PHPBB.com)? Something else?

Would Google list my forum pages?

Google doesn't seem to have any problems listing forum postings from WWWBoard (scriptarchive.com) bulletin board system

Ashwin

2:34 pm on Aug 6, 2002 (gmt 0)

WebmasterWorld Senior Member ciml is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Google seems quite happy to list a few thousand results [google.com].
3:17 pm on Aug 6, 2002 (gmt 0)

WebmasterWorld Administrator rogerd is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Yeah, if Brett ever starts making blue fuzzy widgets he's gonna make a ton of money with those rankings! ;)
4:22 pm on Aug 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



LOL! That answers my question - I need BestBBS !
4:28 pm on Aug 6, 2002 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Ya, there is something in the Google algo that limits the total number of pages indexed on any system. Who can blame them?

I also made many changes here in june that reworked many urls. The total number of potential static urls went up about 3 fold due to the new multipart thread system. I also got rid of all the cgi links that were laying around on the page - which meant some mass 404's for Google these last few crawls.

2:43 am on Aug 7, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Is the WW system free software? I notice it's been steadily improved, which suggests it's at least modifiable.

I'm thinking of putting some discussion boards onto my own site.

2:48 am on Aug 7, 2002 (gmt 0)

WebmasterWorld Senior Member rcjordan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



>WW system free software?

The buzz is that it's going to be "Enterprise Level" pricing ...if Brett ever gets around to canning it.

3:06 am on Aug 7, 2002 (gmt 0)

WebmasterWorld Administrator mack is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Well lets face it, we all put it through it's paces here on Webmasterworld and it stands up very well. Think of the number of users,posts,pages and searches it delivers every day... very good and reliable software. :)
3:16 am on Aug 7, 2002 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Two other threads about it:
[webmasterworld.com...]
[webmasterworld.com...]
5:22 am on Aug 7, 2002 (gmt 0)

WebmasterWorld Senior Member vitaplease is a WebmasterWorld Top Contributor of All Time 10+ Year Member



vibgyor79,

I was also wondering about that in this thread [webmasterworld.com] . Mainly because Google would be the ideal way to find old subjects/threads.

As far as I can see, there are about 235.000 messages (see the sum at the bottom of the forum index; [webmasterworld.com...] ).

I have never heard Brett post the total number of threads, but in this thread: [webmasterworld.com...] Brett mentions average number of messages and threads a day. Lets say 12 messages per thread as average over the years.

That would mean nearly 20.000 threads or 75% are missing at this moment in the Google index.

Lower Pagerank'ed sites have had deeper and fuller indexing, maybe the least internally crosslinked threads have been dropped? Or would Google decide to do less deep crawling on forums?

Then there is Freshness or the frequent spidering and short-indexing of a select number of pages:

Ever since Brett has added a shortlist of the latest participated threads on the member profiles, you can sometimes find Webmasterworld more often when searching in Google for topical subjects, as every once in a while Google decides to refresh-index moderators profiles. (the profile pages of moderators get a new link towards them with every new thread, as the moderator names always are on top of the thread and as far as I am concerned, recent new inbound links is a key criterium for freshness).
Try this for example Greetings from Inktomi [google.nl] . (this example will only last for a short while)

I guess the best remedy for fuller indexing will be Googeguy having a search experience trying to find some of his own threads back again ;)

Added: Then there is also ATW..28.813 listings within webmasterworld.com.. [alltheweb.com]