Help me spec new server

[Jobos]

I'm looking to spec up a new server to replace our ProLiant DL380p Gen8 that is running as VM host (9 VMs). It was bought in June 2016 from ICT Direct and all in all I've been very happy with it. It came with 3 years of HP warranty left on it which was useful as some of the HDs failed over the 3 years but HP sent new drives next day. The server has 2 X Xeon E5-2670 2.60GHz, 72GB memory and 2 X 146GB as RAID 1 for OS, 5 X 300GB RAID 5 for VMs and 1 300GB hot spare. There is 100GB free on the host but I want more storage this time around.

 

I would like SSDs or a mix of SSD and spindle drives if cost is too high. If using SSDs would I RAID them as I've done in the past? HT wants to buy new to get a long warranty but at the end of the day it all comes down to budget so might end up buying used again.

[Katy]

We just replaced our 3 host + SAN VMWare cluster with 2 x HP DL360 g9 (dual 12 core Xeon, 144GB RAM) from ICT Direct, we weren't using SSD (went for 4 x 4TB SAS plus 2 x 200GB SSD) but all in cost us about £2k per server with the 3 year warranty. Not worth the extra it'd have cost to get that spec new just for an extra 2 years on the warranty. @BKGarry did something similar but with SSDs I think which puts the cost up but not by a lot.

 

We set them up as a hyper-converged Hyper-V cluster so presented the disks as just basic disks to the OS which then took care of keeping one as a spare, and mirrors the data over to the other host too.

[dhicks]

cost us about £2k per server with the 3 year warranty. Not worth the extra it'd have cost to get that spec new just for an extra 2 years on the warranty.

 

For comparison, I think we spent around £8,000 on a new version of something similar - fancier NVMe SSDs as the fast tier for storage spaces and 6 x 8TB drives as the main storage, but ultimatly doing a very similar job for around four times the price. Of course, if you're having everyone move to cloud-based storage anyway (Google Drive / O365 / etc) the read/write performance of any local storage probably isn't that important anyway - with a good, low-latency, high-bandwidth Internet connection we're finding we can write files to Google Drive faster than we can to an SD card locally.

[Jobos]

We just replaced our 3 host + SAN VMWare cluster with 2 x HP DL360 g9 (dual 12 core Xeon, 144GB RAM) from ICT Direct, we weren't using SSD (went for 4 x 4TB SAS plus 2 x 200GB SSD)

 

Thanks for the info. What RAID did you use on the SAS drives and I assume it was RAID 1 on the SSDs?

[Jobos]

For comparison, I think we spent around £8,000 on a new version of something similar - fancier NVMe SSDs as the fast tier for storage spaces and 6 x 8TB drives as the main storage, but ultimatly doing a very similar job for around four times the price. Of course, if you're having everyone move to cloud-based storage anyway (Google Drive / O365 / etc) the read/write performance of any local storage probably isn't that important anyway - with a good, low-latency, high-bandwidth Internet connection we're finding we can write files to Google Drive faster than we can to an SD card locally.

 

Cloud storage is something I've not started but is on my list. I think the most we have ever paid of the new server was £5000 but that was a few years ago. The DL380p cost us £1500 which I thought at the time was value for money.

[Katy]

Thanks for the info. What RAID did you use on the SAS drives and I assume it was RAID 1 on the SSDs?

Didn't use any RAID, the SSDs only have the hypervisor on them and we have two nodes which can each run the full load. S2D handles the disk resilience by allocating a spare disk and mirroring all data to the other node in the cluster.

[dhicks]

Didn't use any RAID, the SSDs only have the hypervisor on them and we have two nodes which can each run the full load. S2D handles the disk resilience by allocating a spare disk and mirroring all data to the other node in the cluster.

 

Same here - you just hand storage spaces a bunch of disks, it sorts itself out from there. It's probably worth adding that you should have an HBA card or a RAID card with appropriate pass-through features, otherwise the RAID controller just gets in the way and slows things down. Storage Spaces can use faster disks as a cache-like mechanism to improve read/write performance - give it a couple of 1TB SSDs and all your read/writes will be done at SSD speed, improving performance. I think on our server we have a RAID-1 array for the (small) SSD boot volume, then the NVMe SSDs and 8TB drives are on the HBA controller. You can also get the same functionality all-in-one in a modern NAS - we have a 12-drive QNAP device (around £2,000 new) that, given a couple of NVMe drives and left to get on with it, presents a single fast storage array.

 

In theory, and part of the reason we bought new, is you want hardware that is certified to work with Storage Spaces so you have support if there's any issues. However, as Katy seems to be finding, in practice any reasonable bit of hardware will do and you can save a fair amount buying pre-used.

Edited December 2, 2020 by dhicks

[Jobos]

I'd never heard of S2D until now and had to look it up which shows one of the benefits of working in a primary . I think I'll be sticking with RAID for the time being, no point in upsetting the apple cart with unfamiliar technology.

[Jcx500]

We've just got a second hyper v host (poweredge r640) from ETB-Tech

[ICTDirect_Dave]

FYI we can do fancy-pants NVMe SSD Servers too

 

Refurbished Dell PowerEdge R630 NVMe 1U Server - ICT Direct

[lmgtfy]

We just replaced our 3 host + SAN VMWare cluster with 2 x HP DL360 g9 (dual 12 core Xeon, 144GB RAM) from ICT Direct, we weren't using SSD (went for 4 x 4TB SAS plus 2 x 200GB SSD) but all in cost us about £2k per server with the 3 year warranty. Not worth the extra it'd have cost to get that spec new just for an extra 2 years on the warranty. @BKGarry did something similar but with SSDs I think which puts the cost up but not by a lot.

 

We set them up as a hyper-converged Hyper-V cluster so presented the disks as just basic disks to the OS which then took care of keeping one as a spare, and mirrors the data over to the other host too.

Loving your blog Katy, will you be doing a peice on S2D setup I'd would find that very useful if you did get a chance to discuss your setup etc.

 

Cheers

[Katy]

Loving your blog Katy, will you be doing a peice on S2D setup I'd would find that very useful if you did get a chance to discuss your setup etc.

 

Cheers

 

Thanks I might do over Xmas if I get time, although my setup is slightly different as there's no SSD in the S2D pool (long story short, on my DL360's with the large form factor drives, the SSDs sit in one of the expansion bays but connect straight into the motherboard rather than via a hotswap backplane, S2D refused to touch them because of this) - still works well enough as a cluster with replicated storage, and pulling the power on one host, everything restarts on the other fine.

[Alastairb25]

Does anyone have Crystal Disk Mark results on the various setups mentioned?

[lmgtfy]

Thanks I might do over Xmas if I get time, although my setup is slightly different as there's no SSD in the S2D pool (long story short, on my DL360's with the large form factor drives, the SSDs sit in one of the expansion bays but connect straight into the motherboard rather than via a hotswap backplane, S2D refused to touch them because of this) - still works well enough as a cluster with replicated storage, and pulling the power on one host, everything restarts on the other fine.

I would find it really useful but know how busy everyone is now with remote learning etc nowadays. I know so many schools invest in expressive SANs when perhaps it's not really necessary anymore. Also interested in disk speeds results, don't want a lot do I [emoji16] [emoji1787]

[Jobos]

So it's time I got moving on buying the server as the head keeps nagging me to spend the money before the accounts are closed end of February.

 

The specs of the current server are in the first post but I'm unsure what to look for when browsing the web and don't get me started on the Dell site. The servers there have that many options you would think you are buying a battleship!

 

I have up to £5K to spend and I would like the server to have around 4TB of storage in RAID 10 plus a spare drive. I've never used SAN or S2D so best to keep with what I know and use RAID.

 

Warranty wise I would like a 5 year if possible.

 

Any recommendations?

[Steve21]

I know you said about new and long warranty, but for the level of storage you're asking and the price, have you considered that one Dave mentioned? It'll be running what 3.2TB NVMe in RAID10, as well as 3.6TB~ 10K SAS

 

I'm sure with the spare 1.6k~ you could get an extra 2 years warranty and even some more SSD/NVME if you wanted

 

If you did want new new though, what sort of items did you want? Like 10Gb networking? RAM? etc etc, there's lots of Dell deals on most the time but depends on spec etc as to what model I'd suggest

 

Steve

[ICTDirect_Dave]

I know you said about new and long warranty, but for the level of storage you're asking and the price, have you considered that one Dave mentioned? It'll be running what 3.2TB NVMe in RAID10, as well as 3.6TB~ 10K SAS

 

I'm sure with the spare 1.6k~ you could get an extra 2 years warranty and even some more SSD/NVME if you wanted

 

If you did want new new though, what sort of items did you want? Like 10Gb networking? RAM? etc etc, there's lots of Dell deals on most the time but depends on spec etc as to what model I'd suggest

 

Steve

 

We can definitely do 5 Year warranties

[IntelligenceHS_Andy]

I would go for the HPE DL380 Gen9 or HPE DL360 Gen9. I love the HPE Servers because I love the ILO interface and it's always easy to configure. Sometimes I get confused with Dell; I've never been a fan of the iDRAC's or the RAID Controllers, but I guess it's all down to personal preference.

 

As for spec, I'd go with the Intel Xeon E5-2600v3 over the v4 as the v4's are still pretty expensive as well as the RAM and you don't get much more in terms of performance.

 

RAID 1 with 2x SATA SSD's for the O/S and then 6x or 8x SAS Enterprise SSD's for the VM DataStore in RAID-6.

Edited January 22, 2021 by elsiegee40

[Katy]

+1 for the HPE kit - iLO Advanced is well worth the price, think it was about £60 per server from Dave? Can't remember, definitely "okay I'd be stupid not to" kinda price. HTML5 remote console and remote power control.

[ICTDirect_Dave]

+1 for the HPE kit - iLO Advanced is well worth the price, think it was about £60 per server from Dave? Can't remember, definitely "okay I'd be stupid not to" kinda price. HTML5 remote console and remote power control.

 

iLo advanced is pretty great, that's for sure.

So is the iDRAC enterprise in the Dell servers. Definitely worth having!

 

I make sure to include them in all our standard spec machines now and recommend them to anyone making a custom build.

[Katy]

I would find it really useful but know how busy everyone is now with remote learning etc nowadays. I know so many schools invest in expressive SANs when perhaps it's not really necessary anymore. Also interested in disk speeds results, don't want a lot do I [emoji16] [emoji1787]

Bit later than I hoped but here's the post on the 2 node Hyper-V cluster I set up (with 9th gen HPE DL360s from Dave) https://katystech.blog/2021/01/wac-hyperv-cluster-s2d/

 

Not got much on disk speeds, I've pulled off the graph for the clustered storage volume showing iops, latency etc for the month - we don't run anything very intensive on our cluster as all file storage is in OneDrive, combined with everyone being off site, it's barely doing anything but we've not really noticed any difference swapping from a 3 node plus SAN VMWare cluster to this. We aren't even using SSD on the hosts, we're on 4TB SAS drives (very low budget for this project as it was more of a surprise "oh no 2 of the 3 hosts have died" than a planned move)

 

[lmgtfy]

Bit later than I hoped but here's the post on the 2 node Hyper-V cluster I set up (with 9th gen HPE DL360s from Dave) https://katystech.blog/2021/01/wac-hyperv-cluster-s2d/

 

Not got much on disk speeds, I've pulled off the graph for the clustered storage volume showing iops, latency etc for the month - we don't run anything very intensive on our cluster as all file storage is in OneDrive, combined with everyone being off site, it's barely doing anything but we've not really noticed any difference swapping from a 3 node plus SAN VMWare cluster to this. We aren't even using SSD on the hosts, we're on 4TB SAS drives (very low budget for this project as it was more of a surprise "oh no 2 of the 3 hosts have died" than a planned move)

 

[ATTACH=CONFIG]60548[/ATTACH]

Thanks very much I'll have a good read now [emoji106]

[lmgtfy]

Bit later than I hoped but here's the post on the 2 node Hyper-V cluster I set up (with 9th gen HPE DL360s from Dave) https://katystech.blog/2021/01/wac-hyperv-cluster-s2d/

 

Not got much on disk speeds, I've pulled off the graph for the clustered storage volume showing iops, latency etc for the month - we don't run anything very intensive on our cluster as all file storage is in OneDrive, combined with everyone being off site, it's barely doing anything but we've not really noticed any difference swapping from a 3 node plus SAN VMWare cluster to this. We aren't even using SSD on the hosts, we're on 4TB SAS drives (very low budget for this project as it was more of a surprise "oh no 2 of the 3 hosts have died" than a planned move)

 

[ATTACH=CONFIG]60548[/ATTACH]

Great read by the way, it must have taken a very long time to write this up and I do really appreciate it [emoji106]

[Katy]

Great read by the way, it must have taken a very long time to write this up and I do really appreciate it [emoji106]

 

I only had old hardware to use for the write-up... I must have spent 8 hours shouting at disk controllers 😂 when I did it on new hardware back in September it was easy.

[Jobos]

I know you said about new and long warranty, but for the level of storage you're asking and the price, have you considered that one Dave mentioned? It'll be running what 3.2TB NVMe in RAID10, as well as 3.6TB~ 10K SAS

 

I'm sure with the spare 1.6k~ you could get an extra 2 years warranty and even some more SSD/NVME if you wanted

 

If you did want new new though, what sort of items did you want? Like 10Gb networking? RAM? etc etc, there's lots of Dell deals on most the time but depends on spec etc as to what model I'd suggest

 

Steve

 

I have and also looked at the HP 360 that @ICTDirect_Dave has. Both have enough storage although the Dell 630 would run the VMs very much faster because of the NVMe, but I have a feeling the Dell 630 is somewhat overkill.

 

I suppose in the end it comes down to how much budget you have.