Help with storage build

[dayzd]

So, technical people, once again I need some help.

 

I need some more storage space for backups at my job. We can't afford to go to a named vendor for a proper SAN, so I'm looking to build a Samba/SMB-based 50+TB storage device to hold block-level VM backups (from Veeam, if anyone cares to know).

 

I can probably afford to buy either 24x 4TB disks or 18x 6TB disks, but not 24x 6TB disks. With these I'd have 96 or 108TB of raw storage that by default, my brain wants to use in a RAID10 equivalent to get me 48/54TB-ish usable space.

 

What I'm looking for is advice on how to set up the disks. I don't want to use Windows and ideally there'd be no hardware controller requirement - I'd prefer the array was portable if I ever need to connect it up to a different machine or change hardware. I'm relatively familiar with Linux (mostly Ubuntu) and have a little experience with ZFS, so I'm not afraid to get my hands dirty...

 

• Should I consider a different filesystem/array layout?
  
• Is ZFS a good choice for long term, backup storage?
  
• I've heard people talk about Ceph, but I know little about it. Should I look at it for this purpose?
  
• I'm not fully versed in Unraid, either. Would that make sense here?
  

Given a pile of relatively large disks, how would you get the most out of them, when all-out speed is not your priority and reliability and longevity is?

 

Thanks in advance or your suggestions!

[dmj]

If this is simply for backups, I would probably use bogstandard LVM (with VDO if you need deduplication https://www.redhat.com/en/blog/look-vdo-new-linux-compression-layer) with XFS over the top. You can do most of the things that you need for backup such as daily/hourly snapshots etc with LVM.

 

ZFS is a good option but it's still not terribly well supported in Linux, my feeling is that you'd get better community (and commercial) support with LVM.

 

Ceph is awesome, but its really for clustered filesystems, it's not what you need here. If you need simultaneous writes then glusterfs is another option. If you needed a hyperconverged storage system across multiple clouds/sites then Ceph with CephFS.

 

Personally I'd just go with Centos > LVM/VDO/XFS/Samba and NFS - quick and easy, but there are plenty of OS/s around for this sort of thing if your not confident.

Edited October 11, 2019 by dmj

[mavhc]

I've used zfs on linux for years now, no problems. If you want reliability and longevity of your data you need checksums on your data and your metadata.

 

The only 2 downsides of zfs are you can't grow a raidz array (but you can add another and expand your pool to use both), and it slows down if it gets really full.

[dhicks]

We can't afford to go to a named vendor for a proper SAN, so I'm looking to build a Samba/SMB-based 50+TB storage device to hold block-level VM backups (from Veeam, if anyone cares to know).

 

Your best option for an OS might be something like FreeNAS:

 

https://www.freenas.org/

 

That should have sensible defaukts for dealing with large disk arrays.

 

How do you plan to have Veeam use the storage? We wound up connecting our NAS storage via ISCSI to the Windows server running Veeam and formatting it as ReFS, which Veeam can seemingly make good use of.

 

If you have a Windows server running Veeam, might an option be to move that server to the new hardware you are building, so it simply has direct access to the disk array?