School Virtualization Project

[laser402]

I would hope the RAID10 was Striped Mirrors (Better Performance Boost)

 

Has anyone used RAID50? Could help me squeeze some extra storage in for the budget!

 

Dell are coming to do a Equallogic Demo. From what ive read the controllers are always in Active/Passive mode, never Active/Active, So the talk about 16 disks being too much for one controller must be aload of rubbish! Im still not sure about the lack of GbE ports.... How is your Sun S7000 doing Network wise Duke? What are your peaks etc.

[apaton]

Technical adv/dis of solutions discussed on here.

 

Sun S7120

Advanages

• Innovative use of SSD/Flash
  
• Capacity/Performance
  
• iSCSI/CIFS/NFS/FC + More
  
• Compression/Dedup
  

 

Disadvantaged

• Cost high when including support, gone of the days of the BOGOF promos.
  
• Single Controller
  

 

Dell Equallogic

Advantages

• Dedicated iSCSI solution
  
• Support for VMware vStorage API in firmware 5.x
  
• Dual Controllers, Active/Passive
  
• Scalability by adding additional units.
  
• Can mix SATA,SAS & SSD.
  

 

Disadvantaged

• iSCSI only no NAS
  
• PS4000 scales upto two units, then need purchase PS6000.
  

 

HP Lefthand

Advantages

• Dedicated iSCSI solution
  
• Scalability by adding additional units.
  
• Network RAID, excellent solution for campus wide HA solutions.
  

 

Disadvantaged

• iSCSI only no NAS
  
• Single Controller, need two units for Network RAID to achieve redundancy.
  
• Unable to mix SAS/SATA drives(nodes) in the same storage group.
  

[apaton]

I will have to look up some disk IO specs for the controllers, havent looked too much into all of our current usages as I cant imagine its anything big.

...

Putting all of this onto a new RAID10 10/15K 12/16 Disk Array should easily be sufficient?

 

Yes, but its always ideal to check perfmon so give you some stats.

 

There are a number of IOPS metrics for Exchange, range form 0.2 - 1.0+ IOPS per mailbox, depending on mail box type/usage and version of Exchange.

 

IOPS per disk, RAID Controllers/Cache will improve performance.

SAS disk, is approx 180-210 IOPS

SATA disk, is approx 80-100 IOPS

[apaton]

So the talk about 16 disks being too much for one controller must be aload of rubbish!

 

I agree the Dell Equallogic Active/Passive controller can cope with 16 disks, the 2 x 1Gb iSCSI connections would saturate first.

 

Is 2 x 1Gb iSCSI connections enough bandwidth, usually yes. Applications like Exchange/Ms SQL require low latency (Disk Service Time) as well a bandwidth.

 

Andy

[Duke]

How is your Sun S7000 doing Network wise Duke? What are your peaks etc.

 

I've got a single 1Gb link for CIFS access (two year groups and 600GB of shared resources) with no issues, although I'd likely change this to a 2Gb trunk once the other year groups and staff are on there. Also a single 1Gb link for virtual machines, but this is only about 10 development servers + SCCM currently rather than the whole network. Excluding Exchange and SQL, most of my servers would work okay over a 2Gb trunk (doesn't include file servers which will go straight to CIFS). The S7410 takes expansion cards so I can stick in another 4x 1Gb for network access or put in a 10Gb card that I'll use for replication.

 

To be honest, I hit the limits of the rubbish TCP stack in XP/Srv2003 before I hit the SAN limits...

[laser402]

That sounds promising for the virtual machines however the PS4000's will be doing replication over the same link? (I assume its real time replication?) Still, Ill get them to convince me when they demo it.

[laser402]

Ok, In our latest meeting we have looked at alternate ways of doing things, The problem is our File Server. All of this virtualization/HA/Redundancy will be in vain if this thing fails. We have come to the conclusion that using a SAN (or 2 replicated) with direct CIFS shares may be the best way.

 

Sun S7120 12TB isnt looking too bad now! Although would it handle 800 machines using it for File storage + the VM's over iSCSI/NFS, with 8 GbE ports? Decisions, Decisions....

[Duke]

Sun S7120 12TB isnt looking too bad now! Although would it handle 800 machines using it for File storage + the VM's over iSCSI/NFS, with 8 GbE ports? Decisions, Decisions....

 

Unified Storage suddenly starts to make sense doesn't it?

 

All my file servers (4 of them) will be going on a replicated SAN via CIFS, it also makes them much easier to manage (ahhh... thin provisioned storage, expandable arrays, DTrace!) which is a bonus. I keep prodding management to let me have the second SAN, without it the problem is that you're putting all your eggs in one basket.

 

800 clients via CIFS, 20 VMs via NFS... should be okay, but it depends on what your VMs are actually doing and what your clients are like. If I end up with two classes of 30 each doing a video project that involves streaming and editing raw video off the SAN then that adds a lot more load than just 200 people logging on with large roaming profiles.

 

There are a couple of people on here running the majority of their network off a pair of 7110's - give Andy T and shout and he can probably advise.

 

Chris

[Soulfish]

Ok, In our latest meeting we have looked at alternate ways of doing things, The problem is our File Server. All of this virtualization/HA/Redundancy will be in vain if this thing fails. We have come to the conclusion that using a SAN (or 2 replicated) with direct CIFS shares may be the best way.

 

Sun S7120 12TB isnt looking too bad now! Although would it handle 800 machines using it for File storage + the VM's over iSCSI/NFS, with 8 GbE ports? Decisions, Decisions....

 

Our 7110 (2tb! ) was handling roughly 20 VMs + serving files to roughly 500 computers until recently. It coped (just!), however because we were space constrained we moved to a new EMC SAN. Would have liked one of the new S7000's, but they weren't out at the time and EMC offered stupidly low pricing.

 

Our VMs were connected over 4 x 1gbe links and CIFS over another 4 x 1gbe links. To be honest it was a bit overkill, and would have worked perfectly over half the number of links for each .

[laser402]

Our 7110 (2tb! ) was handling roughly 20 VMs + serving files to roughly 500 computers until recently. It coped (just!), however because we were space constrained we moved to a new EMC SAN. Would have liked one of the new S7000's, but they weren't out at the time and EMC offered stupidly low pricing.

 

Our VMs were connected over 4 x 1gbe links and CIFS over another 4 x 1gbe links. To be honest it was a bit overkill, and would have worked perfectly over half the number of links for each .

 

When you say its coped (just!) Do you mean performance wise or was it more a capacity/storage issue? I assume the 7120's are little more beefy?

 

Our Support Guy also said about leaning away from SD Cards to run ESXi on. He said in his experience it causes the Hypervisor to run slowly compared to HDDs. Dell supply a 1GB SD Card with the R710 for this very purpose... Does anyone has anyone has any issues like this before?

Edited December 16, 2010 by laser402

[john]

If I end up with two classes of 30 each doing a video project that involves streaming and editing raw video off the SAN then that adds a lot more load than just 200 people logging on with large roaming profiles.

 

Oh good someone else who runs the Media Studies directly off the SAN. On my 7110 I could run 12 VMs, 280 PCs logging on (all roaming profiles), using shared areas, home drives, finance program, MIS, Exchange etc and have 12 students editing video live off the SAN before things started to become a bit painful for the video editors.

 

But you have a 7410 don't you Chris thus Im not surprised that mine started to strain a little in terms of end user performance sooner than yours did. I am hoping that now I can spread the load over the 7110 and the 7120

 

We connect ours 2x 1gbe links for CIFS and 2x 1gbe links for the VMs using NFS and iSCSI to a complete seperate LAN. I have now however got a 2nd 4 port card for the SAN so I can consider putting that in to beef it up if needs be, haven't decided if it will go in the 7110 or the 7120 yet.

[Soulfish]

When you say its coped (just!) Do you mean performance wise or was it more a capacity/storage issue? I assume the 7120's are little more beefy?

 

For us it was both. From a 10k SAS drive you can get around 150 IOPs. The 7110 had 14 available for data giving a max sustained IOP load of around 2100. This doesn't take into account things like the L1ARC (RAM Cache) that ZFS makes use of, but it's a reasonable guestimate. We were pushing 2100+ constantly. Do bear in mind that we were running numerous SQL and Exchange servers off of the box. Unfortunately the fact that we were starting to push the limits of what we could expect (to be honest I'd expected it to give out long before it did!), and the fact we just didn't have enough disk space meant we had to get a new SAN. The 7110 was always a temp solution for us, but it lasted a lot longer than expected and performed a lot better than we were expecting.

 

The 7120 is even better, and I believe now has SSDs which greatly help to increase the performance. I'd expect the 7120 would have eaten up the workload of our 7110, asked for seconds and still be hungry for more .

[Duke]

When you say its coped (just!) Do you mean performance wise or was it more a capacity/storage issue? I assume the 7120's are little more beefy?

 

The 7120s are quite different to the 7110s, so although the model number looks like they'll be similar, the 7120 has significantly beefier hardware. Give Phil @ Oracle a buzz about this, the Flash in 7210 will make a big difference.

 

Our Support Guy also said about leaning away from SD Cards to run ESXi on. He said in his experience it causes the Hypervisor to run slowly compared to HDDs. Dell supply a 1GB SD Card with the R710 for this very purpose... Does anyone has anyone has any issues like this before?

 

I've always been recommended to use SD for ESXi and it's what I've specced out on the R710s I plan to buy. The Hypervisor doesn't need a lot of IO once it's loaded into memory, so I can't see disk being a huge bottleneck as long as it's a decent SD card...

 

Oh good someone else who runs the Media Studies directly off the SAN. [...] But you have a 7410 don't you Chris.

 

Thankfully we don't do a full media studies course, but many of our ICT lessons are very media-heavy. Yep, I've got a 7410 with 1x read accelerator and 2x write accelerators which give a bit of a speed boost over the SAS or SATA disks.

 

Bear in mind that although I'm a 850-PC, 22-server, 1600-user school, currently only 2 year groups (userspaces and roaming profiles), 6 test VMs and 2 prod VMs, plus all the shared resources (600+GB) are on there. This has nothing to do with limits on the SAN though, just working around XP bugs and finding time to do data migration.

 

Chris