Welcome to WebmasterWorld Guest from 54.196.244.186

Forum Moderators: lawman

Message Too Old, No Replies

Suggestion

Want to see all posts

     
5:51 am on Jul 25, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 27, 2001
posts:762
votes: 0


Okay, I found that I can look at a profile and see the last few postings someone made. I have a suggestion. Would it be possible to add a button just below that which could show the last hundred postings (or n postings). I've found some people just have interesting things to say but I did not find it easy to work through their posts.

Just a thought.

Richard Lowe

6:46 pm on July 25, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38047
votes: 11


I'm thinking about something like that Rich. It's a pretty intense routine though. I'll put it on the list and bookmark this thread.
6:52 pm on July 25, 2002 (gmt 0)

New User

10+ Year Member

joined:June 5, 2002
posts:13
votes: 0


+1 on this.

Or at least pull up all posts by someone when I plug his/her username into the site search.

6:57 pm on July 25, 2002 (gmt 0)

Administrator

WebmasterWorld Administrator jatar_k is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 24, 2001
posts:15755
votes: 0


that would be interesting

a secondary thought about site search, would it be possible to show a thread date (either last or first post) beside the search result? I usually look at the size to figure the number of posts but if you are looking for an answer on a subject that changes a lot it would be useful to know how old or new the thread was.

6:59 pm on July 25, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member korkus2000 is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 20, 2002
posts:3732
votes: 0


I agree that post date and user post would be very helpful if added to the site search.
5:11 am on July 26, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38047
votes: 11


Site search engine is a hurting unit at the moment. Trying to figure out some new options (it's just so huge now and going to get worse).
5:39 am on July 26, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 4, 2001
posts:997
votes: 0


How is the data stored? Are you using flat files or a real database for the posts?
5:47 am on July 26, 2002 (gmt 0)

Administrator

WebmasterWorld Administrator jatar_k is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 24, 2001
posts:15755
votes: 0


if I remember correctly Brett does everything in flat files.
6:01 am on July 26, 2002 (gmt 0)

Full Member

10+ Year Member

joined:Apr 27, 2002
posts:226
votes: 0


Ok, can I request a translation of "flat files" please? I'm just not following what you mean.

mcg

6:06 am on July 26, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38047
votes: 11


Yes I use flatfiles here. I've got a mysql option worked in, but it is much too slow for our load.

This is a flat file:


jatar_k¦¦no¦¦ip¦1027662426¦if I remember correctly Brett does everything in flat files.
mcguffin¦¦no¦¦ip¦1027663282¦Ok, can I request a translation of &quot;flat files&quot; please? I'm just not following what you mean.<ret>mcg

Flat file just means it is one record per line with the fields separated by a delimiter. Most often the pipe character or a comma.

6:08 am on July 26, 2002 (gmt 0)

Full Member

10+ Year Member

joined:Apr 27, 2002
posts:226
votes: 0


Thanks, Brett!

mcg

6:08 am on July 26, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38047
votes: 11


Or did you mean for the site search Ggrot? I'd love to move that to sql, but there are very few quality sql search engines out there.
6:12 am on July 26, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 4, 2001
posts:997
votes: 0


Sorry. Flat files are generally a basic file that stores all the data. To find the data, one has to search the files. Databases are sophisticated peices of software which allow you to search faster and with more powerful options due to the way they are organized.
6:20 am on July 26, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 4, 2001
posts:997
votes: 0


mySql is actually slower than flat files? How are you doing the flat file search?! Also, are you doing a fulltext index on the fields that you are searching with?

[mysql.com...]

6:49 am on July 26, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38047
votes: 11

Lets back up a minute and make sure we are on the same page. I was referring to flat files for the board here (messages). A forum data structure lends itself best to flat files. They will always be faster than a beefy SQL database.

Why? It's the difference between know where your data is and having to search for your data. Look at the url to this thread. When we need this message, we know the forum (directory), and the file (post number). That allows us in one step to know instantly where this data is at. In turn, it passes it to the file system and we reap all the benefits of caching, and years of optimization.

When you go to a SQL engine, you are doing similar, but it is another layer of code. Not only does it have to figure out where the data is stored at via a index/key file, it has to then call the same file system routines to extract the data out of its own database. Then it has to reroll that data and deliver it to some application.

When you know where your data is at (which is half the battle), a flat file system will always be faster than a database - always.
Thats the qualifier though when know where your data is at.

Searching unknown or huge data sets, that's a whole different situation where dbs come into their own. When you don't know what or where the data you are trying to match is at, an optimized db with a key file or index file short circuts the search and is usually faster than a flat file.

That said, I've yet to see a real quality sql search engine written in Perl. They are very slow and require so much post processing that in the right environment, a good flat file se can still outperform sql (size being the drawback).

I've tried all the perl driven sql engines currently available ([url=http://www.searchtools.com]http://www.searchtools.com[/url]) and currently, there isn't anything that is faster than flat files on both my windows box and linux box when dbs are under 100meg.

Take FDSE for example - it has both a mysql and a flat file option with the flat file option being atleast 50% faster.

Anyway, that's part of the issues I'm facing here with the site search engine. I'm certainly looking for new options, but nothing that looks like "this is it". I've looked at swish and the other stand alones and they aren't what we want here - relevance is a big problem. Even wrote my own here, but the system load is just too huge.

Any recommendations? all ears..

7:06 am on July 26, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 6, 2002
posts:1092
votes: 0


I could tell you the secret to handling large amounts of data in milliseconds but then I would have to kill you. :)

A mixture of databases and flat files. Use the PROs of both systems and avoid the CONs of both systems.