Time to replace our old server, need advice

[RallyTech]

We're currently looking to purchase a new server as our old server is simply running out of space and it's warranty recently ran out.

 

Our old PowerEdge 2900 was an all in one Win 2003 server that did everything DC, DHCP, DNS, Print Server, File Server, WSUS, Anti-Virus, etc. We have about 20-30 users on each day and students use a shared student account. When mobile carts are checked out we can have as high as an additional 60 computers online. Only staff and teachers have folder redirection enabled for My Documents and most computers here are Windows XP.

 

I've been looking into the PowerEdge R610/T610 servers and our budget is roughly 5-6k US $. I'm also interested in taking advantage of virtualization with ESXi or Xenserver free edition as we could use a virtualized Linux server to host a school intranet site and act as a content filter in addition to a virtualized Windows Server.

 

I'm looking for advice on specs and how I should handle virtualization as I'm new to it.

 

So far I have in mind the following.

- x2 Intel® Xeon® E5520, 2.26Ghz, 8M Cache, Turbo, HT, 1066MHz Max Mem

- 16GB Memory (8x2GB), 1333MHz Dual Ranked UDIMMs for 2 Processors, Advanced ECC (Ignore this memory loadout. Keeping this here for reference to original post)

- 24GB Memory (12x2GB),1066MHz Dual Ranked UDIMMs for 2 Processors, Optimized (New RAM spec)

- RAID 1/RAID 5 for H700 or PERC 6/i Controllers (RAID 1 for Virtualized OS/RAID 5 for storage?)

- x2 160GB 7.2K RPM SATA 2.5" Hot Plug Hard Drive (Raid 1, any thoughts on SATA vs SAS? SATA is definitely cheaper so for our small school is SAS really necessary?)

- x3 500GB 7.2K RPM SATA 2.5-in HotPlug Hard Drive (Raid 5)

- iDRAC6 Enterprise

- Broadcom 5709 Dual Port 1GbE NIC w/TOE iSCSI, PCIe-4

- RD1000, Internal SATA Drive Bay for 3.5" Chassis

- QTY2, Internal Removable Hard Disk for RD1000, 160GB Native (should I use this as a backup method? currently use a LTO2 tape backup)

 

I'm pretty open to how I should approach a new server as long as I'm within 5-6k US $ price range. I thought about a PowerVault NX300 for storage and a separate server as well but I'm not really sure if that's necessary for our school so if anyone can enlighten me please do.

 

Update: Added Hard Drive quantities.

Update 2: Added in 24GB of RAM

Edited February 15, 2010 by RallyTech

[SYNACK]

Specs look good for virtualization, not to sure on the Dell components as I usually use HP. I will comment that SAS is much quicker than SATA especially in 2.5" and will make the overall system faster. You could probably get away with SATA but it is not going to be as quick, if going with SATA get as many drives as you can to spread out the load across all of them making the whole setup faster, you may be able to use RAID 1+0 (Spanning + mirroring) or even RAID6 or 5 with SATA to get good performace and sufficient space.

 

I also tend to put all of the drives into one RAID set so as to maximize the flexibility of the solution and allow more drives to work together at any one time for speed and redundancy. You can divide up the storage in the OS using partitions or depending on the controller you can divide up the space created by the single large raid set into virtual drives.

 

You could use the internal 3.5" as a backup staging area for a D2D2T (Disk to disk to tape) backup allowing you to backup the data quickly to the HD then feed it to the slower tape drive without effecting system performace during the day if you have a lot of data.

 

Using external storage is going to be a benifit if you want redundancy for VMs. This would be of use if you were keeping the old server up as a VM host as well, in the case of a failure you could boot the required VMs on the other server quickly. It also can be useful from an expandability sence as multiple servers can hook up to a single data source which may be able to take cheaper, bigger and faster 3.5" drives along with having the capacity to hold more drives in future.

[dhicks]

16GB Memory (8x2GB), 1333MHz Dual Ranked UDIMMs for 2 Processors

 

Is it worth spending a bit more now and getting that up to the maximum 24GB or whatever it is that you can get to within a resonable price? Saves messing around upgrading later.

 

RAID 1/RAID 5 for H700 or PERC 6/i Controllers (RAID 1 for Virtualized OS/RAID 5 for storage?)

 

Sounds sensible. In the past I've always bought the basic Dell server and stuck in a bunch of cheap-as-possible SATA drives from eBay, but I could have sworn there was a post on here a couple of days ago saying that Dell servers are now refusing to work with non-Dell harddrives. I don't know if this holds for drives attached to RAID controllers - maybe purchase both RAID controller and disks separatly to save money?

 

160GB 7.2K RPM SATA 2.5" Hot Plug Hard Drive (Raid 1, any thoughts on SATA vs SAS? SATA is definitely cheaper so for our small school is SAS really necessary?)

 

I'd go for SATA disks for you main storage, SAS simply won't give you the amount of space you'll need - we could easily use up 10TB here, no problem. Adding more RAM to your RAID controller might be helpful (and cheaper) too. We run Windows virtual machines on physical mahines running Linux and the open source version of Xen here. Xen might not be your best choice in the future, as support for the open source version seems a bit variable, maybe have a look at KVM instead.

 

I set up our disks as one large RAID-5 array and split that up with Linux Logical Volume Management (LVM), assigning virtual machines an LVM device each. Using LVM, or a similar technology on a Windows server, lets you take backups of entire VM disks by snapshotting a volume and copying/rsyncing it to a backup server.

 

We use Linux software RAID, mdadm, as we can't afford proper RAID cards. This seems to have worked well enough so far. Increasing performance of software RAID can be done by increasing the size of the read/write cache available - adding more RAM to the machine, so maxing out the RAM to start with might be worth while, and certainly never hurts.

 

You might want to have a separate two-drive RAID-1 SAS array for your system volume and a larger RAID-5 array for your main storage. You could even split up your VM images so they each have a system volume and data volume - system stored on the SAS drives, data on the SATA volume, depending on the performance you need for each VM.

 

Our backup server, with 2.5TB of RAID-5 storage space, is now running OpenSolaris, which might also be a good option. This uses the ZFS file system, which now supports RAID-Z with block-level de-duplication. There are a couple of blog articles from Sun pointing out how good ZFS/RAID-Z/dedupe is, and I get the impression the idea is to make it possible to run decent software RAID. I don't know how having a RAID card combined with RAID-Z helps - maybe it simply provides more bandwidth to each disk? If this is the case, the best idea might be to plug disks in to the RAID card but present them to the system as individual disks still, using RAID-Z in software to do the RAID functionality.

 

QTY2, Internal Removable Hard Disk for RD1000, 160GB Native (should I use this as a backup method? currently use a LTO2 tape backup)

 

160GB doesn't sound like much - you could just get a cheap removeable drive caddy and some 2TB disks off eBay. We're looking at using Amazon S3 for storing some backups, I've not got very far in to investigating exactly how it works yet, though.

 

--

David Hicks

[RallyTech]

About the internal removable drive for backup. I've heard that you have to use Dell branded drives or something to that effect on their servers because it's all somehow locked? Can someone confirm or shed more light because I'd much rather just purchase a group of 1TB WD or Seagate consumer drives for backups.

[SYNACK]

Dell servers block un-Dell HDDs ? The Register

Dell servers block un-Dell HDDs ? El Reg Forums

 

Is the writeup, it looks like you are restricted to Dell branded and certified drives only, you'd probably need to by the hotswap caddys for any other drives anyway which are not cheap.

 

You could try looking at IBM or HP as for the moment these vendors still allow non-branded drives, I would take into account the cost of the caddys though. Your other option, not a good one - would be to use a different branded RAID controller in the server.

 

EG thread here: http://www.edugeek.net/forums/news/50202-dell-blocking-non-dell-hard-disks-now.html

Edited February 15, 2010 by SYNACK

[RallyTech]

Dell servers block un-Dell HDDs ? The Register

 

Is the writeup, it looks like you are restricted to Dell branded and certified drives only, you'd probably need to by the hotswap caddys for any other drives anyway which are not cheap.

 

You could try looking at IBM or HP as for the moment these vendors still allow non-branded drives, I would take into account the cost of the caddys though. Your other option, not a good one - would be to use a different branded RAID controller in the server.

 

Oh that's disappointing.

 

Would this be a good idea? Using an external USB Hard Drive enclosure and swapping out 3.5" 1TB drives nightly? I imagine it would be much slower but this seems like a more practical choice for backups.

[SYNACK]

Oh that's disappointing.

 

Would this be a good idea? Using an external USB Hard Drive enclosure and swapping out 3.5" 1TB drives nightly? I imagine it would be much slower but this seems like a more practical choice for backups.

 

You can do that although USB2 is usually even slower than tape. If you already have the tape I'd use that, we use externals @ 500GB USB2 but it takes so long to backup that overnight does not cut it and we also have more data than that so we are going to external USB3 enclosures which should be fast enough. Used eSATA on one installation for the same reason and while really fast is is simply not as robust as USB. The USB 3 enclosures are comming down in price now two and lots come bundled with a USB3 card which you will need to get full data speed off the new server as I think it only has v2 builtin.

[RallyTech]

I added in 24GB of RAM but at a slower rate to try and make it more affordable. Can anyone else comment on 1066 vs 1333 and UDIMM vs RDIMM and single rank vs dual rank? I've looked at some of the Dell literature and Googled around. My impression is for our situation it might not be a big difference. Does any of this have a significant impact Virtualization performance?

[SYNACK]

I added in 24GB of RAM but at a slower rate to try and make it more affordable. Can anyone else comment on 1066 vs 1333 and UDIMM vs RDIMM and single rank vs dual rank? I've looked at some of the Dell literature and Googled around. My impression is for our situation it might not be a big difference. Does any of this have a significant impact Virtualization performance?

 

First off it is not going to matter about using the 1066 memory in the config you posted as the CPU only supports up to that speed anyway.

 

As to the config it depends on how many chips you are using as to what is better. RDIMMs allow you to have more memory and perform better when you have more than one chip per channel (6 sticks on a DP system) they also have better error checking. The UDIMMs are faster if you are only using 6 sticks but have no error checking and slow down as you add more than one stick per channel due to power loading - at 2 sticks per channel it will slow 1333 to 1066 and at 3 it will drop to 800mhz. I think this is waht is being reffered to by ranking above. Having more ram in a single channel is benifitial though especially if already running at 1066 as it allows for interleaving (splitting accross two chips) which can make things a bit quicker. If you are using only 2 sticks of UDIMM per channel (12 sticks) then it should not really create much performace impact at all.

 

As for speed differences in VMs if you were to go for a faster CPU with 1333 support and 1333 memory then there would be a slight speed increase. How large depends entirely on the workload that the server is being exposed to. If it has lots of apps that hit the memory a lot (huge cached DBs, data processing, large web caches etc.) then the extra speed could be quite worthwhile however it is unlikely to be worth the additional expence unless you plan on utalising it in the above way.

 

This post is an awesome read and explains the new RAM architecture really well:

Dell TechCenter - Dell TechCenter - DELL COMMUNITY

Totally worth a read for anyone looking at new xeon 5000 series based servers.

[dhicks]

The USB 3 enclosures are comming down in price now two and lots come bundled with a USB3 card which you will need to get full data speed off the new server

 

Good point, USB 3 looks worth a try. I think that thread referenced above mentioned using a non-Dell RAID controller for non-Dell disks. I suppose this all depends on how much you're bothered about warranties and so on - we've never really given them much thought, I figure pretty much any computer these days is just a bunch of interchangeable parts, if one part breaks you can just go buy another.

 

--

David Hicks

[RallyTech]

if one part breaks you can just go buy another

 

Ahhh that would be great if I could but it might be a looong time (3-5 years till the next server) before our school makes any other significant hardware purchase. I'm going to get all I can in now and anything that breaks will be covered under warranty.

 

On a side note I have to comment that this forum is great. It's such an awesome resource, I would think there would be more people from US schools here.

[davidchaps]

Rallytech, I'm building a T610 this week to run ESXi and have exactly the questions you did in Feb. Assume you system is running now ? What spec did you end up with, how is it running and anything you would do differently ? Thanks David Chaps

[synaesthesia]

For information the RD1000 is a standard RDX hdd cartridge system however they are indeed firmware locked to Dell. So buy without and get a separate internal SATA rdx from HP/Tandberg which use any compatible RDX cartridge which are literally almost half the price of the Dells.

[Cache]

For information the RD1000 is a standard RDX hdd cartridge system however they are indeed firmware locked to Dell. So buy without and get a separate internal SATA rdx from HP/Tandberg which use any compatible RDX cartridge which are literally almost half the price of the Dells.

 

I bought an Imation 160GB cartridge about two weeks ago which has been working quite happily in a RD1000 in a Dell T310 server.

[synaesthesia]

Ooh, that's useful to know - wonder if the Dell ones work otherwise. The above information was courtesy of Dell themselves. I had to ask as the prices they were asking over and above anyone else (HP/Tandberg/Imation etc) made me fall of my chair in disgust! I assume you haven't done anything to the RD1000 like a f/w flash?

[Cache]

Nope, the server was one I installed myself using 2008R2 and the Dell drivers/tools that were included with the drive. Not changed anything else on the config or checked for firmware upgrades.

 

I did do a lot of searching before purchasing it and nothing I found said that it wouldn't work, the imation site mentioned the Dell RD1000 which what made me plump for it in the end. Imation - RDX® HDD Cartridges - on the features tab if that's any use.