Welcome to WebmasterWorld Guest from 107.20.104.110

Forum Moderators: lawman

Message Too Old, No Replies

Google & Webmasterworld.com

Why doesn't google index all WebmasterWorld.com pages?

     
2:21 pm on Aug 6, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 8, 2002
posts:1157
votes: 0


Hey everybody,

Am I right in assuming that Google does not index all webmasterworld.com forum pages? If yes, why? I see that there are 1000s of plain html files/topics within WebmasterWorld.com but none of them are listed in Google.

If I want to implement a forum in my website, which is the best discussion board software, from Google's "point of view"? YaBB (yet another bulletin board - YabbForum.com)? PHPBB (PHPBB.com)? Something else?

Would Google list my forum pages?

Google doesn't seem to have any problems listing forum postings from WWWBoard (scriptarchive.com) bulletin board system

Ashwin

2:34 pm on Aug 6, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member ciml is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:June 22, 2001
posts:3805
votes: 2


Google seems quite happy to list a few thousand results [google.com].
3:17 pm on Aug 6, 2002 (gmt 0)

Administrator

WebmasterWorld Administrator rogerd is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Aug 2, 2000
posts:9686
votes: 0


Yeah, if Brett ever starts making blue fuzzy widgets he's gonna make a ton of money with those rankings! ;)
4:22 pm on Aug 6, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 8, 2002
posts:1157
votes: 0


LOL! That answers my question - I need BestBBS !
4:28 pm on Aug 6, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38063
votes: 13


Ya, there is something in the Google algo that limits the total number of pages indexed on any system. Who can blame them?

I also made many changes here in june that reworked many urls. The total number of potential static urls went up about 3 fold due to the new multipart thread system. I also got rid of all the cgi links that were laying around on the page - which meant some mass 404's for Google these last few crawls.

2:43 am on Aug 7, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Oct 27, 2001
posts:784
votes: 0


Is the WW system free software? I notice it's been steadily improved, which suggests it's at least modifiable.

I'm thinking of putting some discussion boards onto my own site.

2:48 am on Aug 7, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member rcjordan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 22, 2000
posts:9138
votes: 0


>WW system free software?

The buzz is that it's going to be "Enterprise Level" pricing ...if Brett ever gets around to canning it.

3:06 am on Aug 7, 2002 (gmt 0)

Moderator from GB 

WebmasterWorld Administrator mack is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 15, 2001
posts:7581
votes: 10


Well lets face it, we all put it through it's paces here on Webmasterworld and it stands up very well. Think of the number of users,posts,pages and searches it delivers every day... very good and reliable software. :)
3:16 am on Aug 7, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38063
votes: 13


Two other threads about it:
[webmasterworld.com...]
[webmasterworld.com...]
5:22 am on Aug 7, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 11, 2001
posts:2725
votes: 0


vibgyor79,

I was also wondering about that in this thread [webmasterworld.com] . Mainly because Google would be the ideal way to find old subjects/threads.

As far as I can see, there are about 235.000 messages (see the sum at the bottom of the forum index; [webmasterworld.com...] ).

I have never heard Brett post the total number of threads, but in this thread: [webmasterworld.com...] Brett mentions average number of messages and threads a day. Lets say 12 messages per thread as average over the years.

That would mean nearly 20.000 threads or 75% are missing at this moment in the Google index.

Lower Pagerank'ed sites have had deeper and fuller indexing, maybe the least internally crosslinked threads have been dropped? Or would Google decide to do less deep crawling on forums?

Then there is Freshness or the frequent spidering and short-indexing of a select number of pages:

Ever since Brett has added a shortlist of the latest participated threads on the member profiles, you can sometimes find Webmasterworld more often when searching in Google for topical subjects, as every once in a while Google decides to refresh-index moderators profiles. (the profile pages of moderators get a new link towards them with every new thread, as the moderator names always are on top of the thread and as far as I am concerned, recent new inbound links is a key criterium for freshness).
Try this for example Greetings from Inktomi [google.nl] . (this example will only last for a short while)

I guess the best remedy for fuller indexing will be Googeguy having a search experience trying to find some of his own threads back again ;)

Added: Then there is also ATW..28.813 listings within webmasterworld.com.. [alltheweb.com]

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members