Sun Storage 7110

[m1ddy]

I was thinking of having the following setup....i could of got it horribly wrong

 

We will be having 2 x 7110s mirrored for DR. Linked to these via a SAN only network would be 2 x Sun 4160's with 32GB RAM. Running on these will be ESX VI3.

 

I was going to create 2 projects on the 7110. One holding the NFS shares for my VM images and 1 holding my shares using CIFS for users documents.

 

I am a bit confused of when exactly I need to use iSCSI for block level access. A couple of the VMs will have databases on them (not SIMS, thats staying stand alone). Do I need to setup a LUN for the VMWare image AND a LUN for the d: with the data on it or can I just get away with having a LUN for D:? and still using NFS for windows. I assume doing it either way I will have to use the windows iSCSI tool.

 

How does that setup sound to you guys?

[RobFuller]

NDMP is definitely the best way to backup these boxes - simply plug in a SCSI or FC card depending on your backup environment (FC cards are more expensive than SCSI but give you greater throughput) and plug into the tape drive /library.

 

Talk to Andy on the tape drives / libraries and I'm confident he will churn out a competitive quote.

 

We have either the standalone LTO drives or the SL24 / 48 (max 2 drives, 24 slots on Sl24 and 4 drives, 48 slots in the SL48). don't forget to order the media and for the libraries you'll need to think about barcodes and numbering. Most of the popular backup applications (Symantec, Legato, Tivoli, plus others) are supported.

 

Cheers,

 

Phil

 

Do these backup units have to be connected directly or can this be pulled over the network to a remote server? I'm assuming this will backup live locked LUNs as well?

 

Thanks,

Rob

[teejay]

I was thinking of having the following setup....i could of got it horribly wrong

 

We will be having 2 x 7110s mirrored for DR. Linked to these via a SAN only network would be 2 x Sun 4160's with 32GB RAM. Running on these will be ESX VI3.

 

I was going to create 2 projects on the 7110. One holding the NFS shares for my VM images and 1 holding my shares using CIFS for users documents.

 

I am a bit confused of when exactly I need to use iSCSI for block level access. A couple of the VMs will have databases on them (not SIMS, thats staying stand alone). Do I need to setup a LUN for the VMWare image AND a LUN for the d: with the data on it or can I just get away with having a LUN for D:? and still using NFS for windows. I assume doing it either way I will have to use the windows iSCSI tool.

 

How does that setup sound to you guys?

 

The 7110's have multiple nics, so you can create seperate nic teams, one for your SAN only network and one for your main network to host your CIFS shares.

 

Your other option would be to use replication, but I haven't played with that yet so can't really comment.

 

Get hold of the simulator and do some playing, it's the best way to understand what you can do.

[sjl]

Thanks for that,

 

I have had the simulator playing with it and it definitely does help.

 

I am going to draw up a diagram of our proposed whole setup and hopefully people could have a look and see if there are any problems with it or amendments to improve it.

 

One thing I need clearing up is the amount of storage you get left with on the 7110 (2TB) - am I right in thinking: -

 

Start with

146GB x 14 Drives = 2044GB

 

Two of the drives are mirrored as the system disks: -

 

2 x 146 = 292GB loss

 

Then if you go with RAID5 Double parity: -

 

~ 2 x 146GB = 292GB loss

 

Then you have one drive as a spare

 

So 146GB loss

 

So you will have about

 

1.3TB worth of usable storage to then split up

 

Is that right?

 

Cheers

[Soulfish]

The 7110 actually has 16 disks - 2 are used for the OS and the other 14 are for you to use for storage . Currently got mine setup with the ZFS raid-6 equivalent giving me 1.4tb of usage storage.

[cookie_monster]

You can config it as RAID 5 as well can't you to give you more like 1.6Tb storage?

[kmount]

Couple of blog posts from us on this that might help.

 

Sun 7110 Storage Layout

 

Sun 7000 Series Space Calculator

[torledo]

Do these backup units have to be connected directly or can this be pulled over the network to a remote server? I'm assuming this will backup live locked LUNs as well?

 

Thanks,

Rob

 

i believe with ndmp, with the appropriate ndmp support/licensing in the backup software your using, you should be able to backup to a media server as an alternative to backing up to direct attached tape.

 

you should consult your backup software documentation to be sure.

[RobFuller]

Thanks, Symantec Backup Exec seems to have the option so will have to see if that’s a possibility.

[cookie_monster]

I can't get the simulator to work on Virtualbox, it starts to boot then says that it can't update the microkernel then restarts. Is there an option that I need to set on the VM or mimimum RAM requirement? or an FAQ even?

[kmount]

How much memory have you given it? (It defaults to a gig when imported if I recall correctly).

[cookie_monster]

Not enought then I'll give it a real amount

[sjl]

Hi,

 

I have just created a fist draft of our proposed solution with the 7110, and 2 x X4140s and knocked it up in Visio.

 

The set up is splitting the 7110 into two using a LUN for our ESXi VMWARE images and a CIFS share for storage.

 

There is a private network for the SAN / iSCSI bit

 

I have left quite a few labels off and also the networking of the management interfaces.

 

I have attached it as an image – any feedback would be appreciated.

 

thanks,

 

Steve.

[teejay]

Hi,

 

I have just created a fist draft of our proposed solution with the 7110, and 2 x X4140s and knocked it up in Visio.

 

The set up is splitting the 7110 into two using a LUN for our ESXi VMWARE images and a CIFS share for storage.

 

There is a private network for the SAN / iSCSI bit

 

I have left quite a few labels off and also the networking of the management interfaces.

 

I have attached it as an image – any feedback would be appreciated.

 

thanks,

 

Steve.

 

Apparently NFS is quicker on these boxes than iSCSI and you can also get better analytics on it. Best speaking to Phil from Sun or Andy from Cutter about the backup side, there are a few gotchas on it.

You may want to do this in stages and check the performance, should be fine though with that amount of VM's.

[Hebdenlad]

All,

 

I have been as ked by many of you how and when we plan to impliment User Quotas on the S7000 range.

 

I can now point you at the OpenSolaris.org website and to a specific public domain reference. Now this relates to ZFS but as you all know we are using ZFS under the covers as the file system for the S7000 so it kind of makes sense that stuff that appears in ZFS gets to appear in the S7000 appliance. I must add though that not all stuff appears straight away and sometimes stuff won't appear but please do not be disheartened. The guys that develop this stuff has brains the size of planets and come up with some really good stuff that makes the S7000 such a cool piece of kit.

 

Anyway, enough rambling.... here's the URL http://arc.opensolaris.org/caselog/PSARC/2009/204/mail

 

Any questions then please come back to me. Don't email the guys on the OpenSolaris page as I'm trying to give a single consistent message back into the development team.

 

Cheers,

 

Phil

[cookie_monster]

I have been as ked by many of you how and when we plan to impliment User Quotas on the S7000 range

 

Is it not possible to use any of the File Server Resource Manager tools built into Windows server for this?

[Duke]

cookie_monster - Unfortunately not, at least not to my knowledge. The files need to be going through and be stored on the Windows server for File Server Resource Manager to be able to manage the quotas, which is fair enough if you think about the way it works. FSRM cannot manage quotas on a remote filesystem, and if you enter a quota path in FSRM for a remote share it'll just tell you it's invalid.

 

The solution to this is to map storage directly to the Windows box via iSCSI and use the Windows box to serve files. As Windows will be the one controlling and writing the files and will see the storage as a local drive then FSRM will be able to manage quotas on that space. However, in doing this you obviously lose a lot of the functionality of the SAN box and introduce a Windows server as a bottleneck and failure point. I'm a big fan of FSRM (see my 7410 thread) so I can fully appreciate why you'd want to make use of it.

 

Phil - I've just had a glance through the OpenSolaris docs and I'll fully admit I haven't got my head around it all, but it appears they're doing quotas based on a user rather than a folder/path/share, is that correct? I can't help but feel a lot of people will find this somewhat confusing, particularly if they're coming from a Windows/FSRM background (which is likely why these quota questions are coming up).

 

Certainly the way I'm used to doing quotas (and this will probably be true for most FSRM users) is something like this:

 

• Set up my 'users' directory as 'X:\Users', and in that directory are folders called Username1, Username2, Username3, etc. for each user account. Depending on how people have things set up, the 'Users' folder might be shared or each user's individual folder might be shared.
  
• In File Server Resource Manager I can create a quota, say 250MB, and add email/eventlog alerts at 90%, 95% and 100%. Some of these may come to me personally as a warning, and some may go to the user who's filling up the space to let them know what's going on.
  
• I can then simply apply that quota to 'X:\Users' and tell it to apply to all existing subfolders and any new subfolders that are created. Obviously it will only apply to top-level subfolders, i.e. the Username1, Username2, etc. and won't apply down onto each subfolder individually inside those.
  
• Each user then happily uses their directory, and when they get to 90% they get an email warning, and when they get to 100% they get another email and can't add any more files. This will apply to whoever is putting files in that directory, so if someone in my team accidentally tries to copy a huge file to someone's documents they'll also be stopped and get an email.
  

 

I'm just slightly concerned that if quotas are based on user IDs rather than folders and paths, we'll lose much of this functionality and will be tied into who 'owns' a folder. This could create much confusion for users who have folders they own in several different places and hit a hard quota limit when they only have 5MB in their 'documents' but have 300MB in a 'resources' folder (where quotas aren't normally used as lots of people access that space).

 

Sorry for the long post! Is there any hope for something that might replicate the kind of functionality we get with FSRM? cookie_monster, is this the type of thing you were talking about too, or am I on my own here?

 

Cheers,

Chris

[teejay]

Right, my brain hurts on this one! After a bit of reading, I think zfs already supports quotas on directories, the problem is that the web interface on the 7000 series doesn't yet implement a way to set this except on the share or the project.

 

The new bit adds the option to enforce quotas by a user or a group on a file system. I could see some uses for that, for instance limiting each department to a certain amount of space in a share.

 

So, I think what we are wanting mainly in schools is for there to be the ability in the Web Interface to add quotas to directories within a share. Can this be checked to see if it is in the roadmap Phil?

[Duke]

The new bit adds the option to enforce quotas by a user or a group on a file system. I could see some uses for that, for instance limiting each department to a certain amount of space in a share.

 

So, I think what we are wanting mainly in schools is for there to be the ability in the Web Interface to add quotas to directories within a share. Can this be checked to see if it is in the roadmap Phil?

 

That's pretty much how I read the OpenSolaris stuff too, and the directory quotas is also exactly what we need here.

 

This bit is a lower priority, but since we're building a wishlist - Is there any chance of some kind of email alerting for quotas in the future? We know the Sun S7000 can do email relays, so it'd be really awesome if the user who exceeded the limit got a customisable 'You have exceeded the allowed quota on folder \\san\share\folder, please delete some files or contact [email protected] for assistance' email. This makes a lot more sense to our users than just an error from Windows saying permission denied.

 

Ever hopeful,

Chris

[cookie_monster]

@ Duke, yes that's what I'm talking about, it seems a bit like the Win2k quotas that were based on volumes and ownership which is nowhere near as comprehensive as FSRM.

I see where you're coming from with your comment about the Windows server being a bottle neck but not as a single point of faliure? Is a single SAN device not also a single point of faliure as well? I'd be more concerned with performance.

 

If I can't manage the quotas effectivly with the SAN then I might have to serve it up as an iSCSI device.

 

 

teejay

So, I think what we are wanting mainly in schools is for there to be the ability in the Web Interface to add quotas to directories within a share. Can this be checked to see if it is in the roadmap Phil?

 

And can these limits/quotas be tied to Active Directory groups.

Edited May 13, 2009 by cookie_monster

[linescanner]

Hi,

 

I have just created a fist draft of our proposed solution with the 7110, and 2 x X4140s and knocked it up in Visio.

 

The set up is splitting the 7110 into two using a LUN for our ESXi VMWARE images and a CIFS share for storage.

 

There is a private network for the SAN / iSCSI bit

 

I have left quite a few labels off and also the networking of the management interfaces.

 

I have attached it as an image – any feedback would be appreciated.

 

thanks,

 

Steve.

 

Steve,

 

this looks fine. I would do some testing against iSCSI and NFS. VMware do rate NFS as a good option, and after discussions with Phil, you should get better analytics using NFS.

 

I have one question about the VM layout. Is one of the DC on your diagram the Forest root ? The reason I ask is that we are nervous of virtualising the forest root as you may end up in a whole world of hurt if you plan to recover a failure (OS) from a snap shot.

 

Just an observation.

 

If you need anymore help/info call me tomorrow.

[Hebdenlad]

Right, my brain hurts on this one! After a bit of reading, I think zfs already supports quotas on directories, the problem is that the web interface on the 7000 series doesn't yet implement a way to set this except on the share or the project.

 

The new bit adds the option to enforce quotas by a user or a group on a file system. I could see some uses for that, for instance limiting each department to a certain amount of space in a share.

 

So, I think what we are wanting mainly in schools is for there to be the ability in the Web Interface to add quotas to directories within a share. Can this be checked to see if it is in the roadmap Phil?

I know it's early days on this in terms of implementation of User Quotas in the S7000. I'll try and find out of there are any screen shots / early release versions of the appliance software I can get hold of to clarify exactly what appears in the GUI and how it is managed at that level.

 

Please bear with me on this. I know that User Quotas has been a biggie for customers (large universities and the like) and they are keen to have them as soon as possible.

 

More later.

 

Cheers,

 

Phil

[Hebdenlad]

Right, my brain hurts on this one! After a bit of reading, I think zfs already supports quotas on directories, the problem is that the web interface on the 7000 series doesn't yet implement a way to set this except on the share or the project.

 

The new bit adds the option to enforce quotas by a user or a group on a file system. I could see some uses for that, for instance limiting each department to a certain amount of space in a share.

 

So, I think what we are wanting mainly in schools is for there to be the ability in the Web Interface to add quotas to directories within a share. Can this be checked to see if it is in the roadmap Phil?

have asked the question of the clever guys in the Fishworks team and await their reply.

 

Cheers,

 

Phil

[cookie_monster]

I think this is worth a look for anyone thinking about SIMS or similar applications.

 

The ability of NFS/CIFS to represent storage appliance as a local file system accessible by clients, allows file shared storage access. However, there is number of applications that do not require file sharing. Moreover, there are applications, whose performance is significantly degraded when accessing data using "file IO" approach, like databases and similar transaction-oriented applications.

 

Our results, generated by running some of industry standard benchmarks, show that iSCSI significantly outperforms NFS for situations when performing streaming, database like accesses and small file transactions.

 

 

It's an interesting read.

 

Performance Comparison of IP Storage protocols (iSCSI, NFS and CIFS)

[Hebdenlad]

I think this is worth a look for anyone thinking about SIMS or similar applications.

 

 

 

 

 

 

It's an interesting read.

 

Performance Comparison of IP Storage protocols (iSCSI, NFS and CIFS)

Having read this article it kind of relates to the S7000 and doesn't relate to it either.

 

In the S7000 we have (model depending) a combination of SSDs for both read and write biased acceleration and 7,200RPM SATA drives for the data. We also have zpool RAID equivalents rather than the traditional RAID as noted in the article.

 

I've seen a lot of customers using NFS for their VM estate with really good performance coming off them and the ability to drill down even further on NFS analytics than iSCSI.

 

Have a look on the Sun Fishworks blogs where there is a wealth of information and best practice from the really clever guys on the Fishworks team. also, just try it and see what performance you get. I'm guessing you'll be pleasantly surprised at how fast these boxes really are.

 

I'm digging out some VMware best practice documents so if anyone wants them private me with your contact details and I'll send them off to you.

 

Cheers,

 

Phil