NAS Server - Ideas?

[Soulfish]

I'm starting to look at our backup strategies now we have a (much) larger SAN and are starting to build up more and more data.

 

Currently I'm liking the idea of using either Zmanda or Vembu Storegrid to carry out backups to disk, and then to replicate to a second backup server and/or specific data up into the "cloud" (Amazon S3 Europe most likely).

 

We currently have a Thecus N8800 filled with 2tb drives that we're using as a backup target, but I'm looking at getting a second NAS server. The question is do we build it ourselves, get a supermicro/hp/dell server with openfiler/freenas or get a specific NAS server (qnap/thecus/netgear etc)?

We'd be wanting to get as much space as possible, as cheaply as possible while still being fairly reliable.

 

Anyone have any suggestions?

[Duke]

If you don't need any of the 'enterprise' features, and you can live without a full product warranty and no 'whole solution' support, and it's essentially not a mission-critical device (i.e. if this box falls over, technically your users aren't going to notice any difference), then I can't see a better "as much space as possible, as cheaply as possible" solution than building it yourself - Petabytes on a budget: How to build cheap cloud storage | Backblaze Blog

 

This is something part of me wants to do, and part of me always shies away from - there's no cheaper way to get big storage, but if a component fails I need to have a way to find a compatible replacement, usually out of my own pocket, no way to extend the warranty or buy support on the whole device (generally speaking), etc.

 

If you can afford it, ensure it's upgradable/expandable, and get enough storage for your needs then I say buy an off-the-shelf product, preferably something that will take extra disk shelves. If you can't... get the screwdriver out.

 

Chris

[Soulfish]

I did think about a backblaze box (I use them for my home backups ), and it's certainly an option. Since all backups are to be replicated between two devices as you say it's not mission critical (unless the main file server dies), and the really important data is going to be held in the cloud somewhere.

 

Now do I want to go and build my own backblaze box, or instead get one of the QNap or Overland devices. Choices choices!

[dhicks]

Anyone have any suggestions?

 

Just in case you missed it, there's another thread just started which is probably relevant:

 

http://www.edugeek.net/forums/hardware/68446-server-chasis-homemade-nas.html

 

--

David Hicks

[plexer]

Build a backblaze

 

Ben

[glennda]

Just a copy of what I posted on the other thread Dhicks has mentioned

 

How about this

1 x Antec Twelve Hundred case £109.67

1 x 4GB (2x2GB) DDR3-1333 CL9 Non ECC £28.39

1 x Gigabyte X58A UD5 1366 ATX £177.87

1 x Quad Core Xeon E5506 2.13GHz LGA1366 4.80GT/sec... £145.36

1 x Gainward NVIDIA GeForce 8400GS Graphics Card 25... £19.99

1 x ANTEC SIGNATURE SG-850 PREMIUM 850W A/PFC CABLE... £141.67

10 x Western Digital 3tb Hard Disk at £1583.3

2 x Chenbro 5 x 3.5" HDD Hot-Swap Bays - Fits in 3 x 5.25" ODD Bays £198

 

All in all not bad and under 2500 - motherboard has 10 sata connections plus also raid support

 

if wanted rack mountable Antec do rack mounted cases they just don't have as many slots as the one mentioned above

[EduTech]

QNAP's are great for what you want to do, well worth looking at mate. I use them all the time and never have any issues

[mac_shinobi]

When I was looking at making one, I didn't get as far as motherboard, CPU, Power supplies etc but I was looking at

 

1 * Norcotec Case, RPC 4224 case to be more specific - Norco Technologies, Inc.

 

To start with I was contemplating the WD Green drives ( 2tb ) but got advised that they weren't brilliant in raid configs and that samsung F4 drives were better due to supporting TLER or CCTL, which I don't think the WD drives support unless they are raid edition drives.

 

Any clarification on which drives to go with ?

 

Raid card wise - I was looking at a rocket raid card

 

Other then that just a case of software which there is open filer or free nas - any thoughts on this as I was hoping to use mine in conjunction or with time machine for my apple mac, intend to use it for more then one apple mac in the near future.

 

To start with I was going to get 10 drives and then add more drives but not sure how I can dynamically add more storage and increase capacity if thats possible, seeing as the drives would be attached to the raid card would I have to erase everything and re do the raid config in order to use all the drives or what exactly ?

[dhicks]

We currently have a Thecus N8800 filled with 2tb drives that we're using as a backup target, but I'm looking at getting a second NAS server.

 

The latest version of the Linux kernel, out yesterday, now supports the Ceph distributed file system, capable of scaling on-the-fly to ridiculous sizes accross multiple servers in a fault-tolerent manner. Might be just the thing if you've got more than one server to utilise.

 

--

David Hicks

[dhicks]

got advised that they weren't brilliant in raid configs and that samsung F4 drives were better due to supporting TLER or CCTL, which I don't think the WD drives support unless they are raid edition drives.

 

The Wikipedia page on TLR is very good:

 

Time-Limited Error Recovery - Wikipedia, the free encyclopedia

 

It seems to imply that all WD drives support TLR, it's just that the "RAID edition" ones have it enabled by default. A utility exists to change this, see Wikipedia for details.

 

A thought: in a RAID array, is a harddrive's local cache doing anything? Surley if you have a decent RAID card with a wodge of cache memory, if there's a cache miss in the RAID's cache then there'll also be a cache miss in the harddrive's local cache, so all the local cache is doing is slowing things down. Is that right, or have I missed something?

 

Raid card wise - I was looking at a rocket raid card

 

You might want to check specifications and reviews to be sure you're getting what you expect. If it's not a full, hardware-based RAID card then it might be expecting a Windows-based driver to do some of the work for it, which probably (although, check) won't have a Linux version available.

 

Other then that just a case of software which there is open filer or free nas - any thoughts on this as I was hoping to use mine in conjunction or with time machine for my apple mac, intend to use it for more then one apple mac in the near future.

 

I used plain Debian with Samba installed to join the machine to the Windows domain and provide a network share for Windows workstations. I don't knwo exactly what Apple's Time Machine does, feature-wise, but I understand one of the main headline features is that it stores all previous versions of your files, stretching back over time. This is actually quite easy to do, you just have a script de-duplicate the filesystem after doing a backup with rsync or whatever.

 

To start with I was going to get 10 drives and then add more drives but not sure how I can dynamically add more storage and increase capacity if thats possible, seeing as the drives would be attached to the raid card would I have to erase everything and re do the raid config in order to use all the drives or what exactly?

 

This is exactly why I prefer mdadm RAID, it allows much more flexibility in re-configuring arrays and messing around adding more disks on. Assuming you're adding a number of disks at a time, you could use Linux's Logical Volume Management on top of whatever RAID implementation (hardware, some driver, mdadm) you use to combine RAID arrays together - each time you add a new set of disks, create a new, separate RAID array with them and get LVM to combine them in to the pool. Alternativly, see Ceph, above, which might be able to do the same trick with local volumes as well as network ones.

 

--

David Hicks

[mac_shinobi]

I saw a lot of people giving Drobo's a thumbs up but then read reviews / youtube videos etc on setting them up and they had a lot of negatives ie when setting it up, if you have it connected via usb, firewire or usb / ethernet ( more then one connection at a time ) it wont allow you to configure it, you have to use the usb connection to set it up / configure it and is a total pita to setup.

 

Once setup it works well but am not sure what throughput is like ( read / write on the drives ) and also setting up iscsi using the ethernet connection - doesnt seem to be many videos on actual experience of this.

 

Although it does work with time machine afaik.

 

Would be nice if time machine added support for NAS type devices, when I tried to do time machine backup several months ago

 

1. It didnt seem to give me much control over what I wanted to backup ie checkboxes for me to select what I want backing up ie system state, my home directory, my pictures, videos, music etc

 

2. I had a free 1tb drive and it still stated I did not have enough space and that I required 4tb of space - the only way I can achieve this is via a NAS type device, even the largest time capsule ( 2tb ) is still not enough.

 

It looks like it was or is trying to backup all the data from all the drives I have, as well as all the user data ( not just my own account but one other account I have ) along with the system state.

 

Even if I got 3 * 2tb drives and put it into a raid config ( software raid ) using disk utility if possible.

[pran]

At the school, we use QNAP for the backend storage and Microsoft's DFS for redundancy

[Soulfish]

Thanks for all the suggestions everyone . Building a Backblaze pod does seem tempting. I've even found a UK group that have fabricated the cases and got all the hard to buy parts (UK Blazers and Custom Electronic Enclosures for Engineers and Designers) but I'm also being tempted by a 36-bay 4U supermicro case (Super Micro Computer, Inc. - Products | Chassis | 4U | SC847A-R1400UB). Hmmm decisions decisions

[dhicks]

I'm also being tempted by a 36-bay 4U supermicro case

 

£1,500 for the chasis, 36x£200 for the harddrives, £1,000 for the RAID card - 108TB of storage for around £10,000. I want one...

 

--

David Hicks

[glennda]

£1,500 for the chasis, 36x£200 for the harddrives, £1,000 for the RAID card - 108TB of storage for around £10,000. I want one...

 

--

David Hicks

 

don't tempt me while i'm working on the budget!

[sted]

£1,500 for the chasis, 36x£200 for the harddrives, £1,000 for the RAID card - 108TB of storage for around £10,000. I want one...

 

--

David Hicks

 

make it 2 so i can mirror one to the other

[john]

£1,500 for the chasis, 36x£200 for the harddrives, £1,000 for the RAID card - 108TB of storage for around £10,000. I want one...

 

--

David Hicks

 

Then get it to perform like Oracle 7000 series storage, have the amazing software and analytics's of it and you've got an amzing product at an amazing price

[Soulfish]

Then get it to perform like Oracle 7000 series storage, have the amazing software and analytics's of it and you've got an amzing product at an amazing price

 

If you stuck in the cost of Nexenta Systems you'd get some of the features. Still need some SSDs however but that would be a potent box

[dhicks]

Then get it to perform like Oracle 7000 series storage, have the amazing software and analytics's of it and you've got an amzing product at an amazing price

 

That 36-caddy chasis and the BackBlaze chassis both seem intended for use as backup devices - they both attach disks via port expanders, so you wind up with multiple disks sharing SATA ports. The BackBlaze seems to be specifically designed to be cheap - it doesn't have any dedicated RAID hardware, just some standard SATA ports. I get the impression that BackBlaze also handle reducndancy by having their data duplicated on multiple hardware devices, which obviously makes sense at the scale they operate at. I'm guessing that the performance of either of the above solutions won't be up to, say, 4 classes of 30 plus 40 staff all simultaniously trying to edit documents - for that you'd need something with a capabale hardware RAID card - around £1,000 for something that could handle a dozen or so disks, hence why I'd go for those Antec 1200 cases with front-loading disk caddies, more suited to the scale of your average school.

 

I'm interested to note that BackBlaze base their system on Debian running mdadm RAID - if it's good enough for an online backup company, it's good enough for a school's backup needs. I'm guessing most schools would be better running a Samba server on top of the RAID arrays, though.

 

What features do Oracle's 7000 series storage servers have over a plain file server that just lets users read and write files?

 

--

David Hicks

[john]

The key to the oracle is performance, analytics and overall product design. Really if you just want a pure fileserver its over-priced, its when you look at the whole package, when you get NFS, SMB, iSCSI, NDMP all in the box, plenty to read on here and once you've had one and used one anything else really isn't as good.

[Soulfish]

As John says the Oracle box is a whole package. It makes use of the same backend software stack as Nexenta (ZFS, Solaris etc) and adds good hardware and a nice gui analytics package. The advantage is the way it makes use of RAM, SSDs and HDs to give the fastest performance possible.

 

If you only wanted performance and weren't scared on the command line you could put together something similar using off-the-shelf hardware and Solaris/ZFS (or FreeBSD/ZFS). Alternatively you could possibly get something similar with Linux/BCache or Linux/Fastcache and one of the newer file systems (something like btrfs once its ready!)

[Soulfish]

£1,500 for the chasis, 36x£200 for the harddrives, £1,000 for the RAID card - 108TB of storage for around £10,000. I want one...

 

--

David Hicks

 

Well just to update you all I've just got some pricing for the supermicro 36-bay chassis putting it at around £1200. Total cost for around 40tb of storage (including a good hardware RAID card, 16gb RAM, E5620 CPU etc) is coming in at about £5000-5500 depending on which parts we end up getting. That was based on using 20 of the 36-bays, leaving some room for expansion

[zag]

I built my own 10tb home media server for under £1,000 here

 

MeediOS • View topic - Media server upgrade

 

It was surprisingly easy and cheap

 

 

I'm going to build a backup at work as well using the same principles.

[Fatmas]

I set up a NAS unit recently as a proof of concept, we only wanted it to snapshot our Xen Server so the specification is very low:

 

Dell 210L

- Pentium 4 3.20Ghz

- 2Gb DDR2 RAM

- 2 x 1 Tb SATA2 HDD's in RAID 1

 

OS: FreeNAS 7.3 Sabanda running from USB flash drive.

 

 

It takes a snapshot no problem and has been used to backup the Staff areas using Symantec Backup Exec (Which we already use for main backups)

 

Total Cost: £0.00 (Had everything lying around - Bargain)

[Arthur]

It seems to imply that all WD drives support TLR, it's just that the "RAID edition" ones have it enabled by default. A utility exists to change this, see Wikipedia for details.

You can no longer enable TLER on the latest WD consumer drives. I think StorageReview has more details on this.

 

Once setup it works well but am not sure what throughput is like ( read / write on the drives ) and also setting up iscsi using the ethernet connection - doesnt seem to be many videos on actual experience of this.

I used to have a Drobo, but the performance was so bad I returned it. The new Drobo FS isn't much better (« this review shows you how to set it up on a Mac too).

 

Here are some benchmarks of my Drobo (connected via FW800 with a three HDDs). The first two are using the standard Microsoft FireWire driver while the third and fourth are with the driver from Unibrain.

 

http://goo.gl/OzSFM

 

http://goo.gl/Bs3El

 

http://goo.gl/B5qkh

 

http://goo.gl/zAYdW