I've been looking at RAID systems just recently, trying to find something suitible for home use and thinking through how I could get our servers to perform better for minimal money. In particular, we have a file server that is very slow at times, and I reckon it's the write performance of its disks that is the main problem. The server itself is a virtual machine, hosted on a box running Ubuntu Linux and Xen, sharing a software-RAID harddrive with a couple of other virtual machines. I think things are going slowly when lots of files are being written to the disk - the disk controller's cache is being filled right up and every disk-write is effectively write-through.
This got me thinking that each of our virtual machines could probably do with its own physical disk, and of course that disk should really be a RAID array of some kind so a disk failure doesn't take down our server. I've just bought myself a small two-disk external SATA RAID 1 unit off eBay. This should be just the ticket for my home server, and it handles auto-rebuilding of the array and such like, but at £200 I doubt it's going to have much by way of a cache.
So my current thinking is this: how do I get a motherboard with four SATA ports to make like a disk controller? Can I get it to use two or three of those ports to control disks and the other to plug into another motherboard, with the second motherboard seeing just a normal SATA disk? The bits of hardware are simple enough to put together - a motherboard, small case and power supply, couple of harddrives in caddies, RAM, eSATA blanking plate, internal USB stick to boot off of - and bang, you have a RAID box. What do I do by way of software? Is there a Linux/FreeBSD/whatever distribution (actually, more probably a kernel...) that someone's hacked around a bit to act like a disk controller?
The other option, of course, is to put the exact same hardware together but use it as a NAS box, using pretty much any basic Linux distribution to drive it. Would it be practical to use such a box as direct-attached storage for a larger server - have the larger server have several gigabit network cards in, plug each of those into one of these NAS boxes, including one for the OS to boot off of? How much latency is likely to be involved (bearing in mind that files are going to be transferred over TCP/IP, maybe over iSCISI, instead of something low-level like SATA)?