BillBSET - 4:56 pm on Jun 20, 2012 (gmt 0)
I built a flat file search engine in perl as a learning tool over decade ago.
It gets so much traffic i am trying to optimize it a bit.
The database file is about 1 megabyte in size and has some small compression applied.
I built a flat file database of files with each search term(s), search returns in one file.
This is not as big as you might think, but it is about 6000 files ...
With a Dictionary file with a list of these files.
This makes the file size to open generally under 80k.
Now to the question.
Is it better to have one large file to open and search or one of the 6000 smaller files to open and search?