nas server setup

[PEO]

Hi gang, I'm setting up a server for kids and staff to store adobe premier and powerpoint files etc

 

hardware:

 

2.21Ghz AMD Athlon 64 X2 Dual

2gb RAM

4 X 256GB SATA DRIVES c: 30Gb partition for server software

 

creat each kid a folder with kids having modify and staff read permissions

 

thing is, its going to be 1TB storage server which will have no backup. ICT coordinator is fine with that but I would like to set up some sort of redundancy if a drive fails.

 

and comments and sugestions much appreciated. cheers

[Guest kerrymoralee9280]

RAID sounds like the answer to me - You can pick up cheap RAID cards for next to nothing nowadays or even use software to do it for you.

 

I would mirror it - Still leaves you half your drive capacity. If this is too little stripe it.

[PEO]

RAID sounds like the answer to me - You can pick up cheap RAID cards for next to nothing nowadays or even use software to do it for you.

 

I would mirror it - Still leaves you half your drive capacity. If this is too little stripe it.

 

yep i was thinking raid. is it possible to get sata controller cards that will support 6 drives? the more drives the better, then i could use raid 5. the mother board has 4 inbuilt sata ports.

 

hmm or could i just use the 4 ports on the board pluss the additional sata controller card

[strawberry]

could buy an external usb and set it to sync data nightly/weekly. unplug it when its not in use. Should protect you from power surges or other power problems that could take out all your hdd's at once.

[Guest kerrymoralee9280]

Sorry Mr Happy, not sure how many drives you can support with a SATA RAID controller.

 

At a guess I would say it is possible to have 6, but I'm no expert...... Anybody else?

[PeterW]

Never had any real issues with the Adaptec 3805 8 port drive hardware RAID card Adaptec - Adaptec RAID 3805 although they have recently launched a new 5805 model.

[PEO]

hmm their are 2 port, 4 port, 8 port.... 8 port sata controler cards cost anything from £250+ christ

[PEO]

Never had any real issues with the Adaptec 3805 8 port drive hardware RAID card Adaptec - Adaptec RAID 3805 although they have recently launched a new 5805 model.

 

not cheap hu

[CyberNerd]

FreeNas will provide you with software RAID.

in many cases this is better than the cheap fakeraid cards IMO.

 

HOWTO: Install FreeNAS - Developer Community

[Domino]

I'll go with that- freenas is well good.

[FN-GM]

Rather than mirroring. I have been told that the cheaper raid cards are not as good and not as reliable.

[torledo]

Rather than mirroring. I have been told that the cheaper raid cards are not as good and not as reliable.

 

The best general puporse RAID cards are those made by LSI logic....oem'ed to the likes of Dell and IBM historically.

 

Adaptec aren't bad, but they have had they're fair share of rubbish cards in the past.

 

Can't say i've looked too much at the cheap raid cards. You can't go wrong with a good sata controller with a couple of internal channels. The issue will be how many disks you can cram into an offtheshelf pedestal or 5u server. Intel used to do a storage server that they crammed 12 sata disks into. That's not uncommon these days, and you've got a choice of installing WSS, freenas or something vendor specific such as ipstors NSS.

 

I'll see if i can dig out the specs on the intel box.

[dhicks]

is it possible to get sata controller cards that will support 6 drives?

 

Never mind controller cards, you can get motherboards that support 8 on-board SATA ports. As pointed out, FreeNAS (or OpenFiler, or similar) can do software RAID for you without bothering with a RAID card - all your machine is going to be be doing is serving files, the CPU is mostly going to be waiting around for disk I/O, there's nothing better for the processor to do than do RAID calculations. Personally, I'm using Ubuntu server as I also want my machines to run Xen for virtualization. Ubuntu Server now has an install-time option to be set up as a Samba server, and creating a RAID array is a one-line command, dead easy.

 

For four disks, RAID 10 might be best - two sets of RAID-1 mirrored drives, then those two sets striped. You'll get redundancy and increased read speed.

 

If you're having trouble cramming drives in to your PCs case, something like this might help:

 

3.5" Drive Backplane - 3 x SAS/SATA I/II HDD - Black - ICYDOCK Model: MB-453IPF-B - Central Hardware (Powered by CubeCart)

 

I've just installed one in our new file server - works a treat (and handles SAS drives if you upgrade in the future).

 

If you're considering hardware RAID, you might want to consider an external RAID box. I quite like these ones:

 

RAID - Central Hardware (Powered by CubeCart)

 

I have a two-disk unit at home which seamlessly handles one disk being yanked out whilst running. Fan noise is quite loud, mind.

 

--

David Hicks

Edited April 4, 2008 by dhicks

[torledo]

@dhicks...

 

can you point me in a direction of a barebones server system that has theese 8 onboard ports. I like the concept, but are you not restricted in terms of what raid levels are available in the software. I see not a lot wrong with raid 10 from a fault tolerance perspective but i thing write performance is going to suck. I suppose it is up to the application as to whethe that is an issue

 

Call me old fashioned but i like to have as many spindles as possibe for both performance and fault tolerance, which for me brings up the problem of how many disks a system can accomodate physically.

 

That intel box i was talking about does do 12 SATA bit its a little expensive sans disks. I'd rather spend either very little or a lot on a storage system. An in the middle solution price wise like the intel i would try and avoid if at all possible.

[dhicks]

can you point me in a direction of a barebones server system that has theese 8 onboard ports.

 

This post on AVForums has some model numbers. I seem to remember checking the Gigabyte homepage, they have several boards with 8 SATA ports:

 

Motherboard with 8 Sata Ports ? - AVForums

 

I like the concept, but are you not restricted in terms of what raid levels are available in the software.

 

Linux's mdadm will handle RAID levels 0,1,4,5 and 6, and you can use already-created RAID arrays as block devices in further RAID arrays for things like RAID 10, 01 or 50.

 

I see not a lot wrong with raid 10 from a fault tolerance perspective but i thing write performance is going to suck.

 

What we need is a link to a study somewhere telling us which RAID combination performs best for which operations!

 

I figure write speed can be improved with a decent-sized cache - 4GB of RAM is nothing special these days, and should absorb most smaller write operations, which should cover the majority of things the average user does day-to-day (applications like MS Word are written with write performance in mind - Word often only writes the changes to a file back to disk, minimising the amount of data to be written). You're probably correct about write performance once you write enough data in one go to saturate the cache, though - I guess it's then time to get faster hard drives!

 

Call me old fashioned but i like to have as many spindles as possibe for both performance and fault tolerance, which for me brings up the problem of how many disks a system can accomodate physically.

 

I was hoping for an Antec 900 case and three of those 5-way 3-slot-high hard drive caddies. We had to get new switches instead, though.

 

--

David Hicks

[SYNACK]

I see not a lot wrong with raid 10 from a fault tolerance perspective but i thing write performance is going to suck.

 

I was under the impression that RAID 10 was one of the fastest alternatives for both reading and writing, well dedicated hardware wise, The write times would be doubled at least with a slower software based system.

 

What we need is a link to a study somewhere telling us which RAID combination performs best for which operations!

 

Here's Wikipedas take on it

 

The main ones:

RAID 0 : Disk stripeing, data written half to each disk and read the same way. Quite fast but no fault tolerance

 

RAID 1 : Disk Mirroring, Data written on both disks. Fault tolerant if one disk fails, slower write times and faster read times depending on hardware implementation

 

RAID 3 : Disk stripeing with parity: Uses 3 disks or more, data split between all disks with one providing parity information. Tolerant of a single disk failure, slower reads and writes

 

RAID 5 : Striped set with distributed parity, the same as 3 but with the parity information spread across multiple disks.

 

Stacked RAID:

Addition of RAID levels together to provided increased speed or reliability

 

RAID 10 (1+0) : Mirrored disks that are also striped across multiple drives, needs at least 4 drives. Fast read, slow write if not hardware accelerated, fault tolerant of up to two disks failing so long as only one from each raid 1 subset fails.

[dhicks]

The write times would be doubled at least with a slower software based system.

 

Why?

 

--

David Hicks

[SYNACK]

Why?

 

--

David Hicks

 

Because on a software based system the write operations are not done simultaneously and so when it is being mirrored it must write to one disk and then the other. A hardware based solution will either write both simultaneously or cache it in dedicated memory and write it as soon as possible depending on the quality of the controller. The read operations are less effected as the data can be requested from one disk and then the other while the controller is still waiting for the data to be returned from the first request.

[torledo]

@synack - yes, exactly that....in a software based solution like the one dhicks was describing raid 10 would suck.....

 

for controller based solutions raid 10 are a standard on almost all enteprrise level arrays. albeit very space inefficient, you lost half you're raw capacity....definitely not one for you're expensive 300gig FC drives.

[dhicks]

Because on a software based system the write operations are not done simultaneously and so when it is being mirrored it must write to one disk and then the other. A hardware based solution will either write both simultaneously or cache it in dedicated memory and write it as soon as possible depending on the quality of the controller. The read operations are less effected as the data can be requested from one disk and then the other while the controller is still waiting for the data to be returned from the first request.

 

Ah, hmmm... ... If I've got this right, then a "simultaneous write" to two SATA drives would mean a duplicated DMA request by the processor - the CPU would issue an instruction for the RAM to write directly to location X on both disk 1 and disk 2. Is such a feature exclusive to higher-end RAID controller cards, then, or are there any general-purpose CPUs or southbridge chipsets that can manage this trick by themselves?

 

So does RAID 5 wind up being faster done by software on a modern CPU than by dedicated hardware, then? Modern CPUs are fast, probably several times more so than your average embedded RAID controller CPU, and can calculate XORs at a fair clip, I should have thought. Do RAID chips have other tricks they can use (less complex instructions?... but then modern CPUs are pipelined, you can just keep shovelling in XOR instructions knowing you'll want the results... they even have multiple cores...).

 

I still think the above is going to be rendered pretty much irrelevant by having a large wodge of RAM available as a cache for a system with "normal" write patterns (little chunks of data being written all over the place, rather than big

slabs being written all in one go).

 

Drat, just checked Wikipedia a bit more - looks like I need to explicitly turn on AHCI in the BIOS settings to get the best use out of the harddrives I've just spent all day installing in our new servers... Looks like I'm spending Monday reinstalling OSes!

 

--

David Hicks

[SYNACK]

Ah, hmmm... ... If I've got this right, then a "simultaneous write" to two SATA drives would mean a duplicated DMA request by the processor - the CPU would issue an instruction for the RAM to write directly to location X on both disk 1 and disk 2. Is such a feature exclusive to higher-end RAID controller cards, then, or are there any general-purpose CPUs or southbridge chipsets that can manage this trick by themselves?

The controller itself is most of the issue as most can't write to more than a single disk at a time, they just take turns. I do not know whether any of the newer southbridges have this kind of multiple write feature but it could be worth investigating.

 

So does RAID 5 wind up being faster done by software on a modern CPU than by dedicated hardware, then? Modern CPUs are fast, probably several times more so than your average embedded RAID controller CPU, and can calculate XORs at a fair clip, I should have thought. Do RAID chips have other tricks they can use (less complex instructions?... but then modern CPUs are pipelined, you can just keep shovelling in XOR instructions knowing you'll want the results... they even have multiple cores...).

The thing with the controllers is that they are dedicated real time hardware thats sole responsibility is to manage the disks and as such they should never to busy to deal with a hard drive operation like queuing up another write or read. Some also do the actual calculations in hardware so that the whole parity and splitting process can be performed in a single clock cycle of the controller. Disks as a rule are slow and so even though the computers CPU may be able to perform the operations just as quick or quicker in some cases won't help that much because it is still limited by the disks. The only way to squease more speed out of them is to have a setup that can provided commands and data as soon as it can be handled by the disk and spreading the load across as many disks as you can to increase the overall throughput. These devices also speed things up by changing the read/write command order, grouping read and write operations that will occur in the same area of the disk together so as to do them more efficiently. A technology similar to this has been implemented in Seagate drives for a while now called NCQ (Native Command Queuing) which can speed up certain operations on a 7200rpm drive to be comparable to a 10000rpm drive.

 

I still think the above is going to be rendered pretty much irrelevant by having a large wodge of RAM available as a cache for a system with "normal" write patterns (little chunks of data being written all over the place, rather than big

slabs being written all in one go).

 

The large RAM cache should mitigate a lot of the issues so long as it is configured just right, if it keeps things around in cache to long or pre-loads to much it could fill up quite quickly. It will all depend on your load, the type ans size of the files and how they are used. If you have 10 users hit the box to each play a different large movie file the drives maximum throughput may not be enough to sustain the transfer of all of the files into the cache for distribution. So long as its not under large simultaneous loads like the example above I think it should cope fine.

Edited April 4, 2008 by SYNACK

[dhicks]

The controller itself is most of the issue as most can't write to more than a single disk at a time, they just take turns. I do not know whether any of the newer southbridges have this kind of multiple write feature but it could be worth investigating.

 

Don't suppose you know which ones do, do you (my guess is that answer is going to be "the most expensive ones"...)?

 

The thing with the controllers is that they are dedicated real time hardware thats sole responsibility is to manage the disks and as such they should never to busy to deal with a hard drive operation like queuing up another write or read.

 

I'm assuming that a "general purpose" CPU is being used in a dedicated (and well-written!) NAS/SAN/whatever device of some kind, so it's not going to have to do anything but deal with disk I/O and communicating with whatever sends it data either.

 

Some also do the actual calculations in hardware so that the whole parity and splitting process can be performed in a single clock cycle of the controller.

 

If you mean a CPU operation that takes two blocks of memory, A and B, and produces a block of memory C such that each element is the XOR of each of the corresponding elements in A and B, then I should think most CPUs can handle that by now (they must do, surely?).

 

The only way to squease more speed out of them is to have a setup that can provided commands and data as soon as it can be handled by the disk and spreading the load across as many disks as you can to increase the overall throughput.

 

Oh, indeed. Right: so what do we reckon is faster, then: RAID 10 done by some kind of device that can manage identical writes to two disks at once, or RAID 50?

 

A technology similar to this has been implemented in Seagate drives for a while now called NCQ (Native Command Queuing) which can speed up certain operations on a 7200rpm drive to be comparable to a 10000rpm drive.

 

...But which is, seemingly, generally disabled (AHCI turned off) by default on modern motherboards due to compatibility issues. This means that the servers I've just installed will want reinstalling come Monday. Drat.

 

--

David Hicks

[SYNACK]

Don't suppose you know which ones do, do you (my guess is that answer is going to be "the most expensive ones"...)?

 

Not a clue unfortunately, as I said I'm not even sure that this is supported in standard desktop controllers. If it is I would say that it would be through one of the ones that has some form of RAID built in which in general only work with Windows. They do a lot of their work in CPU but some of them may be able to employ certain tricks like this that are usually found in larger controllers.

 

I'm assuming that a "general purpose" CPU is being used in a dedicated (and well-written!) NAS/SAN/whatever device of some kind, so it's not going to have to do anything but deal with disk I/O and communicating with whatever sends it data either.

 

In a NAS or SAN there will be a dedicated GP CPU floating around to handle the external requests and all of the management tasks. This however is backed up by one or usually multiple heavy duty controllers that handle the disks themselves.

 

If you mean a CPU operation that takes two blocks of memory, A and B, and produces a block of memory C such that each element is the XOR of each of the corresponding elements in A and B, then I should think most CPUs can handle that by now (they must do, surely?).

 

Kind of be on a much larger scale, think more along the lines of a GPU with multiple stream processors. You have a stream of commands that come in and chucked into a queue, these are prioritized by time, proximity to other read/writes and the status (full/empty) of the cache at the time). Given the design this will all be happening while a parallel set of circuits grab the optimized commands and throw them at the disks. In the case of RAID 5 it will take the data in blocks (64, 128 or even 512bit blocks), break it up and generate a parity all in the one clock cycle (many more on a 32bit CPU). Push all of this to the buffer for writing and then possibly put a read operation in the queue to check the consistency of the data on the disk. All of these things can be run in parallel when put together by really clever people and so the equivalent system with GP CPUs would require a dedicated quad core with some pretty intelligent hardware in between the CPU and the controller to give all of the CPUs access to the disks at the same time.

 

Oh, indeed. Right: so what do we reckon is faster, then: RAID 10 done by some kind of device that can manage identical writes to two disks at once, or RAID 50?

RAID 10 is faster as the operations involved are far simpler even for dedicated hardware. This is why it is probably the most common form used in large organizations. RAID 50 will give you more avalible disk space from the same disks but is slower.

 

...But which is, seemingly, generally disabled (AHCI turned off) by default on modern motherboards due to compatibility issues. This means that the servers I've just installed will want reinstalling come Monday. Drat.

 

Unfortunately NCQ only really got picked up by Seagate, it is a subset of the technology that is put into larger RAID controllers but limited in scope to each drive individually. I know that it does run alright on a fair few motherboards (with BIOS upgrades usually) but my experience with it is in single drive, non RAID systems so I have no idea how it deals with software based RAID. It is from my experience a worthwhile technology and it is a shame that it has not been taken up more universally.

Edited April 5, 2008 by SYNACK
Fixed typing error: replaced 126 with 128

[torledo]

@synack - were you a storage professional in a previous life Very impressive knowledge.

 

between yourself, dmccoy and dhicks i don't think there's a storage question that can't be answered on here.

[SYNACK]

@synack - were you a storage professional in a previous life Very impressive knowledge.

 

between yourself, dmccoy and dhicks i don't think there's a storage question that can't be answered on here.

 

Thanks torledo, the degree I did was in computer engineering and so we learnt about all sorts of stuff involving embedded logic and custom electronics. That and I have had to read a few spec sheets for RAID controllers in the past.

 

If all of us on the site put our knowledge and experience together I think we have a pretty fair chance at answering most questions which is what makes this forum so useful