Fileserver Replacement

[Shuriken1]

Evening all,

 

(Apologies if this has been covered before. I did have a quick look through the shortlist it gave me, hopefully you won't consider this a waste of your time to read. Thanks.)

 

I'm in the position where I need to replace our fileserver. We have only one at the moment serving approx 800 users (students and staff) and the HDDs cannot cope, they're hitting 100% disk time for about 15mins between lessons, causing the odd network location not found error, all sorts of things. The current setup is Raid 5 for the data drive, they're something-capacity 7200rpm SATA drives - not the most fileserver-worthy drives imo. After that it's 1x dual core 3.4ghz (ish) xeon and 4Gb of RAM, neither of which get stressed at present.

 

I was thinking of replacing it with two servers but money dictates that I don't think that will be possible, so, my plan is this:

 

2u server with 12 3.5" drive bays in the front.

2x E5620 2.4Ghz

8Gb RAM

2x small 15k SAS drives for system (internal).

2x 1Gb NICs teamed together.

 

Then for the main data drives, two RAID 10 arrangements:

 

1x 600gb RAID 10 comprising of 4x 300Gb 15k SAS drives - for staff profiles, shares etc.

1x 1.2Tb TAID 10 comprising of 8x 300Gb 15k SAS drives - for students.

 

Shared data will probably stay on the old fileserver as I think it can probably manage that.

 

You can probably guess that I'm trying my hardest to avoid the HDD bottlenecks we're currently experiencing. Have I got a good plan here? Will those 15k RAID 10s give enough speed increase over the single RAID 5 we have now. Is splitting the staff and student data onto two RAID arrays going to help? I presume it will as one won't overload the other.

 

I'm open to any feedback on this one, have I got the right idea? The last thing I want is to buy this and it make no difference, so I'm looking for some confirmation that this is a workable plan really.

 

Your time (and feedback) is greatly appreciated.

 

Thanks,

 

Shuriken1.

[_Adam_]

File servers typically do not need much in the way of processing power or memory as you pointed out, so not sure you'll see much difference here. Where you will is moving to RAID10, RAID5 is poor at writing speeds. If you're only using 1 controller I would probably go one larger RAID for both staff/students as you're effectively giving staff 2 spindles. One controller per RAID would see performance slightly better. Make sure you pick a quality RAID controller. Also servers typically have 4 NICs these days, I would team all 4.

 

You don't mention what type of profile you're running or how much is stored on the server so it's hard to know if your server spec would be okay or not.

 

Is there a chance to RAID10 the current server for staff and move students to the new server?

[Shuriken1]

Thanks a lot for your reply Adam,

 

I think we'll only have the one controller. Please excuse my ignorance but you mention effectively giving staff only 2 spindles, could you explain that further please? I'm not sure the reasoning behind your suggestion (I'm sure you're right, I just don't know why).

 

The staff are running roaming profiles at the moment, they're a mess. I'm trying to sort them out as well but it's kind of merging into one horrible job. Students have a mandatory profile. I've spent a long time since I started trying to get the staff profiles under control, it's turning out a big problem. It's the roaming profiles that cause most of the load I'm sure, I'm also sure that when i do finally sort them it will drastically reduce the load on the fileserver. Maybe even to a point where the current server could manage but I don't think so. I've got the go ahead for a new server so I want to make the most of it.

 

The data drive on the fileserver has about 330Gb of stuff on it, thats staff, students and some common shares - basically everything data wise. I'll be leaving the shared stuff on the old server and just moving the staff and students.

 

Unfortunately I can't RAID10 the current server as it only has the 3 disks in it and no space for more. Thanks for the heads up on the NICs, I'll be sure to team as many as I can!

 

Thanks for your help so far, I'd appreciate anything further you can add.

 

Shuriken1.

[_Adam_]

You may know this already, but just to help explain what I mean.

 

I will go back in time to when the U320 controllers were around, this meant bascially 320MB/sec transfer. No single disk can achieve this, they are around 80MB/sec. By adding disks together (increasing spindles) a controller could read/write faster up to 320MB. At a minimum this is RAID0, and RAID5 reads the same way but adds a parity bit for error checking. RAID10 stripes and mirrors so with 4 disks you're writing on 2 spindles and these are being mirrored to the other 2. Using the same figures above it would be up to 160MB/sec.

 

So when I mentioned just putting staff on a large RAID to make use of more spindles this is why.

 

The strain on your file server is 800 people hitting it the same time, typical for schools.

 

Based on what I am running I would try this:

Students - Mandatory profile with folder redirection on Documents only

Staff - Roaming profile with full folder redirection and sync to offline files on logout

 

This should stop a ton of files going over your network at the same time. Office documents open on the network, a local temp file is made while you're editting, and then it's save back to the network. So there shouldn't be any introduced slowness.

 

Of course there is more to consider, kinda still talking big picture.

[SYNACK]

I also agree, one large RAID so that the full amount of bandwidth is avalible to the disks (all read/writes shared out between 12 disks). If you need to split the areas logicly just use partitions. I would also make sure that your RAID controller is a good quality one with at least 512MB of battery backed Cache which will also add to the speed of the IO.

 

I'd probably add a couple more 1GB ports to team up also depending on the number of clients or look at a SAN setup and some extra NICs to attach to your current servers.

Edited March 31, 2010 by SYNACK

[Duke]

Have you considered anything in the way of networked storage (SAN/NAS) rather than a standard Windows server? I only ask as they tend to be purpose-built devices that will give you better performance through dedicated flash storage, RAM, storage-oriented operating system, etc. NetApp have things like Flash card modules to accelerate performance and Sun use hybrid storage pools with SATA and SSD disks to give really high IOPS while maintaining good storage space. They're also generally much easier to expand in the future rather than having to buy a new server with bigger disks.

 

If you're sticking with Windows boxes then one way to improve things is to spread the load of user spaces and roaming profiles over several servers, which is what we did. A couple of year groups on each server, three servers in total with 10k or 15k drives, keep the naming of directories simple by using DFS, job done.

 

Just a few thoughts.

 

Chris

[teejay]

Yeah, to give you an idea, we have replaced 7 windows 2003 file servers with 1 unified storage box. The file servers were struggling with the load, the unified storage handles it with no problems at all.

[Shuriken1]

Thanks for the further explanation Adam and everyone else for their thoughts.

 

I've pretty much got the setup you suggest there Adam, only without the offline file sync. Well I say got, I'm in the process of moving the staff over to profiles that are fully redirected rather than just docs and desktop.

 

Thanks for confirming the point about one big raid SYNACK, I think that sounds the way to go - especially with Adam's further explanation. I'll check the momry on the Raid controler as you've suggested - thank you. It's just the standard Intel one I think but I will check.

 

I have looked at SANs, yeah, last night in fact. It's a great setup and would be great but I don't think we're quite there yet as a school. From what I gather it would mean migrating from a dedicated server arrangment, ie servers with their own resources to a centralised system, that being the SAN. Great idea, but I don't think we can afford the switchover costs - so it's just a new fileserver for now.

 

I was also thinking of getting two smaller servers like you suggest but again, I don't think we can afford it, hence the single server approach. Do you think a two server setup with the load spread would be significantly better than the idea I'm planning?

 

I think that's everything, thanks a lot,

 

Shuriken1.

[teejay]

Can you give us an idea of the size of school we are talking about here, number of computers etc

[Duke]

Do you think a two server setup with the load spread would be significantly better than the idea I'm planning?

 

It's definitely possible, but very much depends on what you buy, exactly where the bottlenecks are, type of files being transferred, and how much you've got to spend. We saw big benefits when we moved to three dedicated file servers, although will ironically now be moving back to one SAN (with failover).

 

Any idea what your budget is and whether there's any leeway to get more funding if you can sell the idea as a long term solution?

 

Cheers,

Chris

[Shuriken1]

Well we've just drafted the budget for next year and this server replacement isn't on it as I wasn't aware of it back when we did the budget. I've got £5k from the bursar to sort this out.

 

I was just thinking more about the SAN suggestion since my last post and I supposes theres no reason we can't run the old servers as they are with the SAN replacing the fileserver, so it's in place. Then as we replace future servers they can use the SAN too. Does that sound plausible?

 

The bottlenecks are the hard disks at the moment, due to poor staff roaming profiles that are large and cause heavy load on the server.

 

I doubt if I can get much more money even as a long-term solution. I could maybe get £7-8k for this, if its a definite long-term solution as I can pull money from another project. Unfortunately my knowledge is limited when it comes to SANs.

 

teejay: It's a secondary school with about 750 students and about 100 users. 275 computers and 80 odd laptops for staff. What did you replace your 7 fileservers with?

 

Duke: I'd be interested to know more about your SAN plans, unfortunately I haven't really found any decent examples that suit my situation, anything SAN and they seem to start talking massive datacentres and huge disk arrays - a bit overkill for me. I do like the expandability (that a word?) of the SAN setup though.

 

Thanks guys, as ever, appreciate all of your help,

 

Shuriken1.

[Duke]

I was just thinking more about the SAN suggestion since my last post and I supposes theres no reason we can't run the old servers as they are with the SAN replacing the fileserver, so it's in place. Then as we replace future servers they can use the SAN too. Does that sound plausible?

 

The bottlenecks are the hard disks at the moment, due to poor staff roaming profiles that are large and cause heavy load on the server.

 

Yep, a SAN/NAS that provides CIFS (Windows shares) and AD authentication should be able to replace any Windows file server unless you have any particularly unusual requirements. You can also map space on the SAN (via iSCSI or CIFS) to an actual Windows server so it can make use of the space if needed. If you ever decide to go the virtualisation route then your SAN can also act as the shared storage (via iSCSI or NFS) so your actual 'servers' are stored on the SAN too.

 

£5k is a little tight for a good SAN but £8k is doable. However, I'm cautious of over-selling the idea to you as I don't want to suggest something that's really going to be bigger and more enterprise than you need, when one or two Windows servers with good disks might actually do the job fine.

 

teejay and I are using similar products and I know he's got way more load on his than I have with no issues so they really are very capable and flexible boxes.

 

I'll PM you my email address and direct line in a minute, if you want to give me a call/email tomorrow or next week then I'm happy to spare a few minutes to chat about SAN/NAS in general and how we use ours, and put you in touch with a few people who may be able to help further and give you an idea of prices.

 

Cheers,

Chris

[teejay]

We have moved all our file storage onto Sun S7000 Unified Storage and also use it for server virtualisation. If you want a chat about it PM me. Also if you go on the Oracle website you can download a virtual machine of the Sun Unified Storage software, but make sure you update it to the latest version.

They do models which with edu discount would fit your budget depending on your storage size requirements (they do data de-dupe on these now so your storage requirements will be less).

[Guest monkeyx]

We are looking into Nexenta The Nexenta Project - Community edition and Nexenta Systems for fully supported version.

 

Sort of very much like the S7000 series minus the price tag

 

Openfiler Openfiler — Openfiler - Open Source Storage Management Appliance also has good support for iscsi and CIFS

 

Tim

[Duke]

We are looking into Nexenta The Nexenta Project - Community edition and Nexenta Systems for fully supported version.

 

Cool, looks interesting! We wanted the whole package (e.g. hardware and software, and the warranty/support that goes with it) but if you're happy to roll on your own storage that looks nice.

[Guest monkeyx]

There are some UK parnters that offer a full system. I am talking to a couple of them to see what they can offer.

 

Tim

[Duke]

There are some UK parnters that offer a full system. I am talking to a couple of them to see what they can offer.

 

Awesome, I like the peace of mind that comes with a fully warranty once you start talking about TB's of storage.

[Shuriken1]

Thanks a lot for your info guys, I may well take you up on that offer Duke/teejay - thank you. I've just disocvered another school in the area already running a SAN so I think I'm going to go over and check that out before I make any final decisions.

 

Thanks again for your thoughts. I'll update this after I've been to SAN and let you know what I go for.

 

Shuriken1.

[_Adam_]

I have concerns that you would not put yourself far ahead purchasing a SAN with your budget over the original idea of 12 SAS drives in a server.

 

There are a lot of "extras" that will add up quickly in cost and this would result in fewer disks just to afford it.

 

If you're talking budget and going iSCSI, what speed are you looking at? Are you planning a dedicated switch? Does the switch enable jumbo frames? There is much more to consider.

 

Post what you get offered, I am interested.