Building a better site search?

[Brett_Tabke]

The search engine here on the site is ok, but I want something a bit more robust.

Would it be much of a hassle if, I were to build a regular search engine, but only index files once a day? That would mean that todays messages would not be searchable.

[rcjordan]

I vote MORE robust even if it means not indexing the current day. How hard would the indexing task the cpu? If it's intense and given our international traffic, you're going to have to pick your cron times carefully or you might slow Woz to a crawl.

[tedster]

Brett, I think it's a great idea.

The trade-off of going to once a day indexing in order to add a more robust search would be a big boost. And when dealing with today's posts, the thread I want is usually easy to find, anyway.

[Woz]

>slow Woz to a crawl

Thanks for the kind thought, it does happen sometimes although nothing to do with Brett. The other day when I wanted to post my long editorial on registereing Chinese Domain Names, I typed it all in and hit "POST" and.... nothing..... sat there for 15 minutes to see if it got through but..... nothing..... totally frozen, nothing in, nothing out. Frustrating!

Had to log off and wait 'til the next day to find it hadn't gone through, and repost it.

However, to the question in hand, Brett, if you are going to PostgreSQL, can't you just search the raw database? Or am I missing something?

Onya
Woz

[DaveAtIFG]

>And when dealing with today's posts, the thread I want is usually easy to find, anyway.

Same here, and I'd like to have a better search, go for it.

[oilman]

Works for me - the dailys are easy enough to track

[Brett_Tabke]

re:database search

I am planning for scalability. Who knows, we could end up hosting forums for other sites, or expanding the current offering 10 fold. So that means keeping the database flexiable. Instead of going with one huge table to handle everything, I am going with a table (database) for each forum. A search of the raw board would be one search for each table (currently 25'ish). Combine that with the required post processing by Perl, and it would be too slow (currently running 15 seconds in MySQL windows - too slow).

re:growing old waiting from china.

Sorry for the hassle Woz, that may have been a local problem. The host has been having trouble in peak times. This box is fine most days, but peak hours during the week, it has been unacceptably slow (that is the reason we are preparing to move). There were two times last week when the board was unreachable for 15mins each during peak periods.

[Woz]

>>Sorry for the hassle Woz

Sorry Brett, wasn't clear enough in my post.

It was not your site at fault, it was my connection. I couldn't get enything. Not WmW, SEF, GHF, nothing! Jim posted a message at SEF about a DOS he had which happened at the same time my connection went down. A coincidence I am sure. It seems everything went down at once.

So, no need to apologise (or apologize either),

Onya
Woz