Sun 7410 Unified Storage + SAN/Userspaces Question

Hi folks,

First post, but I've been keeping an eye on the forums as a couple of people I've been working with are on here. I've recently bought a Sun 7410 Unified Storage System with a 22TB J4400 Array and three flash accelerators. I know there has been quite a bit of discussion on the Sun 7000 series, but it's been mainly aimed at the lower end 7110 and 7210 models, so if anyone has any questions about the 7410 then I'll do my best to answer them, although be aware I haven't fully deployed it yet in production so I won't have all the answers!

(Thanks to Andy at Cutter Project and Phil at Sun for their help, very good people to work with!)

On to my related question...

For those of you who have a SAN and Windows users with roaming userspaces/profiles, how have you got them saving directly to the SAN? At present we have three Windows servers with shares for userspaces and profiles, these locations are set in the user account in Active Directory and the 'My Documents' folder is redirected with Group Policy.

I'd like to have these users (1600+ of them) writing directly to the SAN to remove the Windows server overhead and point of failure. However, I'm starting to realise some issues like the ability to batch-create user shares and set permissions on them (easily done with our Windows user account tool), granular control of file and folder permissions, and quota management at the level provided my Microsoft's File Server Resource Manager.

If you're using a SAN with lots of Windows users, how do you create and manage the shares required on a large scale? I realise this isn't a problem with the Sun box (it fully supports AD and CIFS, and does no less than NetApp or EMC kit to my knowledge), but more a 'what's the best practice' question.

Many thanks,
Chris

Hi Chris,

I'm afraid I can't answer any of your questions (we've recently got our 7110 and haven't had a chance to play with it yet!), but I did just want to say that I'm insanely jealous of your 7410 and all that storage :p

How are you finding general performance out of interest?

Hi Soulfish,

The 22TB was a bit overkill, but when you consider that drops to 14TB in RAID6 and a format with two hotswaps, then the room you need for snapshots, it becomes a bit more reasonable! While we've got nowhere near 14TB of data on the network at the moment, I was planning for future expansion and using the SAN to store virtual machines for our virtualisation project.

I'd previously looked at NetApp, EMC, Dell, SANMelody and others for storage solutions, but at the promo pricing the Sun box was very hard to argue with. I already have a 4TB NetApp box so was seriously considering going with NetApp, but the Sun 7410 was basically half the price of a comparable NetApp 2050, plus all the licensing and OS is free!

I've only done some basic performance testing so far, and at the moment it's almost identical to a Windows box which simply tells me the bottleneck is somewhere else on the network. I'm looking into sorting out a better testing system as DTrace certainly hasn't shown the Sun box reaching any limits!

For those who might be interested and can offer any input, our long-term plan is:

• Pair of mirrored Sun 7410's
• Four Dell servers (Dual Nehalem Xeons + 36GB RAM) running VMware ESX (not yet tested, although Xen is very nice) in one location
• Six or eight older Dell servers running VMware ESX elsewhere on site for failover
• Virtualise the majority of our current 22 servers

Cheers,
Chris

@Duke: I think Andy had asked me to tell you how I did this... and I then forgot :o

I found the easiest way was to create shares for the relevant things and asdsign the root full permissions for one user (me). I then created a folder in that share and dropped everything under that, using Windows to set the ACLs on those folders - I already had a script.

I used ROBOCOPY to copy the data off the old shares to the new ones and used the /SEC switch to keep the ACLs intact.

Sounds like a good plan Chris. Might be worth testing Xen - I have it on good authority that it outperforms ESX currently.

Working on an order for a much scaled down version of exactly what you are planning.
Pair of mirrored Sun 7110's
Two Sun Servers (Dual 3.16 Quad Core Xeons + 16GB RAM each) running ESX / XEN
Two older RM Servers (Dual 2.33 Quad Core Xeons + 8GB RAM each) for redundancy.

Let us know how you get on!

Butuz

Ric - Thanks, that sounds good to me! I know that will definitely handle things like our Staff and Subject Resources folders and then we can just change the drive mapping for users.

'My Documents' userspaces are a little more tricky, at present it looks like this:

Share on server as username$
Username has full permissions
Staff have read-only permissions
FSRM sets a hard-limit quota with email alerting at 90%, 95% and 100%
'Profiles' folder is shared, and subfolders for each user with permissions

I'd like to be able to make use of our existing user-creation tool (tools4ever User Management Resource Administrator, not cheap but works brilliantly) if possible to create the shares and permissions, but obviously there's no way for it to create shares or set root-level and share-levels ACLs on the Sun box. I know the Sun OS will take scripts, so that's definitely a possibility. Whatever solution we use needs to be manageable by everyone in my department, who are coming from a mainly Windows-only background.

The easy alternative is to have a Windows 'head' with thin-provisioned storage attached via iSCSI then we just manage everything via the Windows box. It's not ideal as we have another bottleneck and point of failure, but it does make everything simple in a Windows environment!


Butuz - I've got two Xen boxes here at the moment and have been very impressed! I couldn't believe how easy it was to set up compared to Hyper-V. The fact HA just takes a central heartbeat and a tickbox compared to having to completely configure Windows Clustering is brilliant. My only concern is that Xen hasn't got anywhere near as much market share as VMware and the features aren't quite there yet (check out the VMware 2009 roadmap and watch the Fault Tolerance video). I've yet to give ESX a proper test so I'll see how it goes.

It sounds like your plan is very similar to ours in principal so I'd be interested to know how yours goes!

Cheers,
Chris

@Duke: You should still be able to create your homedrives, etc. as you do now. The only difference will be that you won't create a username$ share for each user - simply a users share with subfolders.

There is a script on the wiki that will sort your permissions out (it uses XCACLS) once you have connected to the relevant share.

Profiles can be done in the same way - Profiles share, subfolders and script the ACLs.

The only thing that I haven't sussed yet is how to take advantage of quotas. I have time to figure this out though because of the jump in storage space that I am taking.

I use XenServer too. I'm not sure exactly which features you are after but the main ones are there already and it is still the cheapest option for lice migration of VMs. The HA is also awesomely simple to use as you mention.

If you want to have a chat about things, please give me a shout and try to make it to the conference (although Andy mentioned you had some stuff on to do with the provisioning of the beast).

Hi Ric,

Thanks for your reply, much appreciated. I did a quick test with a 'users' share and a 'profiles' share but had some issues with permissions. Even though through Windows their effective permissions were full control, they still didn't have permission to access \\sunbox\users$\username, but I'm sure this was down to me getting something wrong! If we can get this working then our user creation tool should still be able to create all the userspace and profile folders for each user (around 250+ each year) which will be a big bonus.

We would be okay on the quota front too if required as we'll have way more space than we need. However, with 1600 users the potential for them to fill up their userspaces with large files is worrying, so it's nice to have a cap on them. Not sure if you've ever played with File Server Resource Manager but it's actually a really good tool, but obviously limited to the local Windows server it's on. If Sun could find a way to create something like 'Set quotas on all folders in the top level of this share at: 300MB' rather than just a quota for the share that'd be brilliant.

We're looking at getting Phil and someone from Cutter, plus a colleague at a local school in at the end of the month, so I'm sure we'll be able to figure out a better idea of things then.

Cheers,
Chris

Where are you based Chris?

Butuz

In Wiltshire - Hardenhuish School. :)

Chris

Hi Chris,

I've been testing tools4ever today using the 7000 series simulator.
It works fine to create user folders within a share set up with the Sun web interface.

I would create a profiles share and perhaps a userspace share for each year group. That way the tools4ever script can create all the user folders within the shares and can inherit permissions from the Root ACL. You could then do as Ric suggested and use something like Robocopy to move the existing user folders to the new shares.
This does not help with the quota issue though. I have sent you an email with details of the permissions I used during my testing. I hope it's of some use to you!

Rick

Quote:

---

Originally Posted by Duke

Hi folks,

First post, but I've been keeping an eye on the forums as a couple of people I've been working with are on here. I've recently bought a Sun 7410 Unified Storage System with a 22TB J4400 Array and three flash accelerators. I know there has been quite a bit of discussion on the Sun 7000 series, but it's been mainly aimed at the lower end 7110 and 7210 models, so if anyone has any questions about the 7410 then I'll do my best to answer them, although be aware I haven't fully deployed it yet in production so I won't have all the answers!

(Thanks to Andy at Cutter Project and Phil at Sun for their help, very good people to work with!)

On to my related question...

For those of you who have a SAN and Windows users with roaming userspaces/profiles, how have you got them saving directly to the SAN? At present we have three Windows servers with shares for userspaces and profiles, these locations are set in the user account in Active Directory and the 'My Documents' folder is redirected with Group Policy.

I'd like to have these users (1600+ of them) writing directly to the SAN to remove the Windows server overhead and point of failure. However, I'm starting to realise some issues like the ability to batch-create user shares and set permissions on them (easily done with our Windows user account tool), granular control of file and folder permissions, and quota management at the level provided my Microsoft's File Server Resource Manager.

If you're using a SAN with lots of Windows users, how do you create and manage the shares required on a large scale? I realise this isn't a problem with the Sun box (it fully supports AD and CIFS, and does no less than NetApp or EMC kit to my knowledge), but more a 'what's the best practice' question.

Many thanks,
Chris

---

Hi there "Duke",

more than happy to help on this one if I can?

can you contact me so we can start talking?

I know some of the guys on here have roaming profiles running on their S7000's

Cheers,

Phil :D

Quote:

---

Originally Posted by Duke

Hi Soulfish,

I'd previously looked at NetApp, EMC, Dell, SANMelody and others for storage solutions, but at the promo pricing the Sun box was very hard to argue with. I already have a 4TB NetApp box so was seriously considering going with NetApp, but the Sun 7410 was basically half the price of a comparable NetApp 2050, plus all the licensing and OS is free!

Cheers,
Chris

---

hi chris,

can i ask which netapp box you currently have ? A fas 250 or a fas 270 is it ?

had you basically maxed out the storage capacity on your netapp, also which feature licenses are you running ie snaprestore ?

@Duke: Have you upgraded the 'firmware' image on the box? The latest release(s) have fixed some Windows ACL problems.

OK, got roaming profiles and users set up and live on our Sun 7110.
This is the easiest way of doing it:

Make sure you have the latest update on your Sun box, makes this work a whole lot easier!
Create a Project for the Users, eg Students.
Create a share for each group of users, for instance for each Year Group.
On the CIFS share level ACL set your admin group to full control and give a security group with the users who are going to have their home directories in this sgare all read and write access but don't tike the inheritance options.
In the CIFS Root directory ACL give your admin group full control and the share users group:
Read Data/List Directory
Execute File/Traverse Directory
Read Attributes
Read Extended Attributes

Then, in the share create two folders, Documents and Profiles
On each of these give your Admin group Full Access
Your share users group:
Traverse Folders/Execute File
List Folder/Read Data
Read Attributes
Read Extended Attributes
Create Folders/Append Data
and set this to apply onto this folder only
Add permissions for CREATOR OWNER as everything apart from Change Permissions and Take Ownership and set this to apply onto Subfolders and Files only.

Right, to copy your users across, use:

Code:

---

robocopy \\oldserver\share \\sunbox\share\documents /E /ZB /DCOPY:T /COPY:DATO /W:1 /R:1

---

where \\oldserver\share is the location of your users Documents folders. You can do the same for the profiles if you wish, but I found it best to let the profiles be recreated. You may get a few users complaining about missing favourites etc though!

Couple of caveats on this, first you need to make sure the user is the owner of their Documents folder and their profile folder.

I used /COPY:DATO rather than COPYALL as we found trying to copy across the ACL from the old server wasn't very successful. Using /COPY:DATO just copies across the ownership and the other permissions are inherited from the folder you are copying into.

Hope this helps! We have now copied across 4 year groups without any problems using this method.

PS - I'm extremely jealous of your 7410 setup!