Welcome to WebmasterWorld Guest from 18.210.27.34

Forum Moderators: open

Message Too Old, No Replies

Building a better site search?

     
12:33 am on Nov 4, 2000 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38254
votes: 113


The search engine here on the site is ok, but I want something a bit more robust.

Would it be much of a hassle if, I were to build a regular search engine, but only index files once a day? That would mean that todays messages would not be searchable.

1:52 am on Nov 4, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member rcjordan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 22, 2000
posts:9138
votes: 0


I vote MORE robust even if it means not indexing the current day. How hard would the indexing task the cpu? If it's intense and given our international traffic, you're going to have to pick your cron times carefully or you might slow Woz to a crawl.
6:35 am on Nov 4, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


Brett, I think it's a great idea.

The trade-off of going to once a day indexing in order to add a more robust search would be a big boost. And when dealing with today's posts, the thread I want is usually easy to find, anyway.

Woz

8:25 am on Nov 4, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member woz is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Aug 13, 2000
posts:4823
votes: 0


>slow Woz to a crawl

Thanks for the kind thought, it does happen sometimes although nothing to do with Brett. The other day when I wanted to post my long editorial on registereing Chinese Domain Names, I typed it all in and hit "POST" and.... nothing..... sat there for 15 minutes to see if it got through but..... nothing..... totally frozen, nothing in, nothing out. Frustrating!

Had to log off and wait 'til the next day to find it hadn't gone through, and repost it.

However, to the question in hand, Brett, if you are going to PostgreSQL, can't you just search the raw database? Or am I missing something?

Onya
Woz

3:32 pm on Nov 4, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 21, 1999
posts:2141
votes: 0


>And when dealing with today's posts, the thread I want is usually easy to find, anyway.

Same here, and I'd like to have a better search, go for it.

4:47 pm on Nov 4, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 10, 2000
posts:2151
votes: 0


Works for me - the dailys are easy enough to track
3:54 pm on Nov 5, 2000 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38254
votes: 113


re:database search

I am planning for scalability. Who knows, we could end up hosting forums for other sites, or expanding the current offering 10 fold. So that means keeping the database flexiable. Instead of going with one huge table to handle everything, I am going with a table (database) for each forum. A search of the raw board would be one search for each table (currently 25'ish). Combine that with the required post processing by Perl, and it would be too slow (currently running 15 seconds in MySQL windows - too slow).

re:growing old waiting from china.

Sorry for the hassle Woz, that may have been a local problem. The host has been having trouble in peak times. This box is fine most days, but peak hours during the week, it has been unacceptably slow (that is the reason we are preparing to move). There were two times last week when the board was unreachable for 15mins each during peak periods.

Woz

1:32 am on Nov 6, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member woz is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Aug 13, 2000
posts:4823
votes: 0


>>Sorry for the hassle Woz

Sorry Brett, wasn't clear enough in my post.

It was not your site at fault, it was my connection. I couldn't get enything. Not WmW, SEF, GHF, nothing! Jim posted a message at SEF about a DOS he had which happened at the same time my connection went down. A coincidence I am sure. It seems everything went down at once.

So, no need to apologise (or apologize either),

Onya
Woz