|Webserver with database in local environment|
Would you recommend a 1U server in this case?
| 1:57 pm on Feb 11, 2005 (gmt 0)|
For one of my customers I am currently developing an internal web / dataserver application. This company has several smaller production locations. Each production location generates a lot of data which must be monitored by the local staff. This data is only interesting for the current production, i.e. most data can be discarded at the end of the day. Interesting compressed information is uploaded to a central located server at the end of the day for statistics and history.
We figured out that a dynamic web application based on a SQL type database would be the best way to implement it. Each production location must have its own server because there is not enough bandwidth available between the locations to have one central server which serves incoming and outgoing data streams for all production locations.
One problem this company has, is that they only have one datacenter with adequate staff at their headquarters. So the local system must be easy to manage, idiot proof and if possible fit into the small space left in the 19" rack now used for their router and switches on each location.
I was thinking to use 1U servers, but I have no experience with them. They seem to be less configurable than "normal" servers when it comes to hardware options like raid, number of disks, etc.
- 1,000,000 data items uploaded to the database each day
- Data uploaded at a rate of 15 to 45 items per second
- Database is totally cleaned at the end of each day
- 5 to 10 users are viewing web pages at a rate of 1 to 10 pages per minute
- System will probably be Linux with Apache, PHP and mySQL although no final decision has been made yet
What I would like to know:
- Does anyone have experience with this amount of data on 1U servers compared to larger servers?
- Would normal IDE be good enough, or go for SCSI?
- Any partition scheme ideas for the harddisk that would improve performance?
- Other valuable comments about things I've forgotten?
| 3:08 pm on Feb 11, 2005 (gmt 0)|
* Does anyone have experience with this amount of data on 1U servers compared to larger servers?
* Would normal IDE be good enough, or go for SCSI?
* Any partition scheme ideas for the harddisk that would improve performance?
* Other valuable comments about things I've forgotten?
I would go for SCSI just because it has onboard cache it's also more reliable then IDE
I wouldn't go with partitions so much as an array of 3+ drives in a raid 5 configuration and hot swappable drive bays.
| 3:38 pm on Feb 11, 2005 (gmt 0)|
RAID 5 is not my favorite in this situation as the system will have much more write than read operations. RAID 5 has some overhead on write operations to calculate the parity data. Data is not that valuable that I need a RAID configuration for data safety, speed is more important so RAID 0, 1, or 0+1 are options. Total data stored in the database is between 100 MB and 1 GB, so disk space is no problem.
My technical problem is that - when possible - it all has to fit in one 1U chassis. With 3 disks, I was thinking to use two disks in raid 0 configuration for the data, and the third disk for the OS, log files etc.
| 7:33 pm on Feb 13, 2005 (gmt 0)|
I don't think you'll have any performance issues based on the numbers you posted. A 1U server is still a server, it's just not tall.
However, the problem with 1U's is that they do all the work of a 2U, 3U, 5U....but in much less space. This means your biggest enemy is heat. I don't think I've ever run a 1U unit with more then 2 harddrives in it and come to think of it, I don't even know which manufacturers have such servers. The SCSI drives take up so much space, there is barely any room for a slim dvd/cd drive and a floppy drive :)
I'd have no qualms about using a dual 10,000 SCSI HD 1U server to do the work you need it to do. I'd run RAID 1, definitely agree not RAID 5. RAID 0 is a little better performing then RAID 1, but I really think you'll sleep better with a RAID1 config.
(I just checked Dell and HP, and neither have 1U's with more then 2HDs.)
Hope this helps,
| 7:59 pm on Feb 13, 2005 (gmt 0)|
Thank you for your input. It is good to hear that the amount of data itself is not a problem. Intel is making 1U servers with 3 drives. That's why I was thinking of 3. I didn't find other manufactures though with that drive capacity and with the amount of data it might just be overkill.
Heat can be a serious problem, thank you for mentioning it. The production process of this customer is taking place in an agressive environment so all 19" enclosures are without ventilation to keep the air inside clean. I think I will first setup a test with a heating element inside one of their enclosures to see if temperature stays within reasonable limits. I've read not long ago that heat dissipation was also one of Google's primary concerns with their data centra :)
| 11:38 am on Feb 15, 2005 (gmt 0)|
If you are stuck with 1 drive, use a 15K rpm as disk I/O will be the bottleneck.
With SQL server (Dont know about MySQL) , when you write data, it writes to the transaction log at the same time as the devices, so to get the most benefit from RAID you separate the devices, transaction logs, and SQL Application/OS and may use Raid 0,1, and 5! Sure, overkill for 5-10 users!
1 question, how are you implementing the replication to the main host system with MySQL?
| 1:12 pm on Feb 15, 2005 (gmt 0)|
It is not the amount of users that's worrying me, any server can do that. The database needs to accept the incomming stream of 15 to 45 unique records per second during the day. This data is generated by production machines. The server can not simply say "Hey, I am busy, try again in five minutes", it has to store the data immediately.
We do not replicate to a central MySQL server. The statistical data is stored on a AS/400. Selected data is dumped from MySQL to an ASCII file, and that file is uploaded again in de AS/400.