School Server and infrastructure revamp

[MrWu]

Hi all;

 

I have been given the chance to give our secondary school a revamp and to replace the server room with something more robust. I have been suggested to set aside 30k to replace or add the following:

 

2 x Windows 2008 DCs

1 x Fileserver / Printserver (current storage for teachers and students 1TB)

1 x SIMS Server

1 x DPM 2007/2012 Server

1 x SCCM 2007 / SCCM 2012 Server

1 x Hyper-V (to virtualise old Intranet Server,Impero Server and maybe a Virtual DC

 

We will carry on using our Exchange 2010 (on new hardware) keep the QSAN P300Q iSCSI 12 TB SAN (most likely to the DPM 2007/2012 for backup storage), I have a Tanberg LTO5 library Drive that I'm thinking whether to sell on or keep in the network for archival, rest of old servers are Q6600 with Consumer SATA 7200rpm Drives thats been running the network of 1100 students and 150 staff, have been told by a few suppliers that I need to spend super duper money to handle a potential increase to an extra 150 sixth form students on top of the next term...

 

What do you guys think?

[john]

How many devices will you have on your LAN?

 

If I was starting with that kind of money it would be 3 or 4 servers all based on virtualisation technology and have everything running on them as it will give redundancy and resilliance.

[MrWu]

Hi john,

 

There are 400 PCs at present, all XP except win7 on some admin dept. AV mangement and windows updte control is lacking ...I've budgeted around 24k the servers for my first pass:

 

5k for the 2 DCs

7k for the fileserver / printserver

3k for SIMs server

2.5k for DPM server

2.5k for SCCM server

4k for Hyper V server

 

Also have 15k marked for switch upgrades, and cabling as needed, but depends on an audit and test if cables switches we have existing

 

There is also another fund to extend the wireless coverage around school and students to Bring thier own device to log in, thats involving a Sonicwall Aventail and a couple of terminal servers at the moment.

 

i will have a look at Virtualisation too, i have been qouted 25k for intel multiflex solution, but doesnt cover all the replacement servers.

[kmount]

Can you elaborate a bit on what you currently have hardware / software licensing wise, and also how much data you're using (and over how many spindles)?

 

Also, it would be useful to know what the suppliers have recommended you spend super duper money on for an additional 150 users, I've in the past ran similar numbers (1200+150 staff) on a pair of servers with 3 disks in a raid5 array (oh the good old days...) and it didn't skip a beat but that was before multimedia and other things came along.

 

I, like John would be looking at a virtual environment, probably Hyper-V if you have licensing for it. (or can get it cheaply, otherwise VMware's free esxi version would be good)

 

£30k is a good chunk of cash, spent wisely in conjunction with an understanding of the current deployed hardware (for understanding a sizing on the 'requirement' of the new virtualisation equipment) and any existing 'gripes' from the customers ie, slow speeds, not enough disk space etc could yield quite an improvement for you.

 

 

(Declaration of circumstance, I work for a company who sells stuff, yadda, yadda, etc. Not the purpose of the post.)

[MrWu]

Hi kmount,

 

I havent started for long at the school, give me a few moments i will jot down as much as i can about the school system today for you. I am glad upto 30k is a healthy budget to put aside, and will value any opinions later on :-)

[steve]

Personally I'd virtualise more:

 

2x DCs 2K each

2x Hyper V 6K each (SIMS, SCCM, Print, old Intranet Server,Impero Server and maybe a Virtual DC)

1x DPM 2K (use old SAN for storage)

1x New SAN for Hyper V machines + File storage (12K left for this should get you about 6TB)

 

The Hyper-v machines can have about 100Gb RAM for that money, giving plenty of expansion room. Plus using data-centre licences will give you unlimited licences for extra servers.

[Millgate]

Hi @MrWu,

 

If you require any help from any of the vendors, just say. Eg, if you want any conference calls, trials, technical help, quotes, surveys etc etc.

 

We partner with everyone so whatever you need, I'll be ble to help out

 

Luke

[MrWu]

Thank you Steve;

 

I've come from commercial side of IT, What do you think of HP Proliants? Are they popular with schools?

 

Ok here's a long summary of the Server Network:

 

1 x DCs : GC,DNS,DHCP (Windows 2003 R2 Enterprise, E6850 3GHZ Core 2 Duo, 4GB RAM)

1 x DCs : GC,DNS, Printserver (Windows 2003 R2 Enterprise,Q6600 2.4GHZ Quad Core, 4GB RAM )

6 x Fileservers : Student/Teachers etc (Windows 2003 R2 Enterprise, Q6600 2.4GHZ Quad Core, 4GB RAM) was 7 but one died after powercut

1 x Old Exchange Server: To be migrated (Windows 2003 R2 Enterprise, Q6600 2.4GHZ Quad Core, 8GB RAM)

1 x New Exchange Server (Windows 2008 R2, Xeon E5620 x2 2.4GHz, 16GB RAM)

1 x Remote Desktop Server (Windows 2008 R2, Xeon X3430 2.4GHz, 8GB RAM)

1 x SIMS Server (Windows 2003 R2, Q6600 2.4GHz Quad, 3 GB RAM SQL 2008)

1 x Impero (on a PC)

1 x Intranet ( Server died after powercut , now Virtualised and running on a ML150)

1 x Frog Server

 

1 x QSAN P3000Q iSCSI 12TB (at the moment attached to one of the filservers, wasted I think)

1 x Buffalo TX-RXL280 4TB NAS (used for backup using Backup EXec 12.5)

1 x Tandberg LTO5 12 slot Storage Library (attached to Fileserver, only working as a single tape drive.previous IT Manager bought the wrong SCSI Card)

 

Most of the old servers are over 3 years old, cobbled together and using 4 x SATA 500GB hard drives on RAID 10 (these are 7200rpm consumer jobbies..they often fail)

 

It's a single site network with different buildings attached by fibre and branching off to approx 9 locations via fibre(mostly terminates on 1 Gb Fibre Switches or converters > 24port gigabit switches, very messy so I will need to document and untangle to see what I could do..(In my mind I set aside 15k to update as needed..thats fairly much depending on what I find)

 

I would say compliants mostly being logins a bit slow (I find the disk performance on all the servers atrocious, you click on a folder and it lags for a few seconds) nothing major though, so I was wary about suppliers claiming I need super IOPS Fileservers to make it workable...this solution needs to last 5 years however.

 

I'm more concern that there is no central control of AV, Windows Updates are switched off (It was a leave it alone it won't break situation) and the network hasn't been documented for years, in fact in another thread I mentioned a power cut that took out 2 servers so far and boy that was a test of character for me trying to keep things afloat with no info(the remaining techs have been life savers though)...so UPS propsals are going in too.

 

We have a microsoft school agreement

 

Hope this all helps to paint a picture.

Edited January 7, 2012 by MrWu

[MrWu]

Hi @MrWu,

 

If you require any help from any of the vendors, just say. Eg, if you want any conference calls, trials, technical help, quotes, surveys etc etc.

 

We partner with everyone so whatever you need, I'll be ble to help out

 

Luke

 

Cheers Luke.

 

Will do. :-)

[steve]

We switched to HP Proliants a few years ago and have been impressed with the price / performance. Some suppliers can get extremely good deals via special bids to HP. If you're looking to do HP switches too, you should get a cracking deal for all that equipment.

 

Never really had any issues for support, but the odd ones we have had were dealt with quickly enough.

 

We only have 3 relatively low spec (4gb ram) DCs for a network of 700+ machines with good login speeds. I'd look at your login profiles - are they roaming or mandatory? If roaming then your network and files servers will be taking a battering if the profiles are large.

[MrWu]

Yeah, loved the HP, i ran a DL360 G1 about 8 years old running TS in my old company and as far as i know still going strong.

 

The teachers use roaming which we will change soon with folder redirection instead. Students use roaming with folder redirection.

 

Also, i heard a lot about 10gbe backbones, is 1gb links still adequate? Cost 10gb still very high.

[kmount]

Hi there,

 

Many thanks for the detailed overview of the current environment, that is very helpful indeed.

 

I think you have quite a few options available to you on how you could improve the current situation whilst also looking at the future.

 

I would standardise on HP as Steve mentioned as they are good boxes and it makes sense, my advice would be to look at DL360's or DL380's if you need more disks.

 

I am a stickler in always recommending to keep a physical DC in a virtual environment, some people say I'm stuck in the old days but I don't like all of my eggs in one basket and a physical DC keeps me breathing when the SAN takes a turn for the worse or a 'bad thing' happens and everything goes a bit titsy wobbly and you're left scratching your head with the whole world dead, at least with a single physical dc you can have something.

 

You have a lot of fileservers there for 1100 users, any particular reason you know of or was it simply a cobble together? Do you have access to any i/o details from these boxes as to whether they are being hurt on read or write or are they just generally quite tragic? I'd expect a fairly reasonable SAN would handle this load without too much trauma depending upon how big the profiles are and what kind of 'usertypes' you have with log on / log offs etc simultaneously and what kind of network throughput these boxes are getting combined (ie, if you have 6 of them all on 1GbE links think of the total combined together!).

 

I'd be looking to team the nics together in sets and segregating the LAN and iSCSI/Storage sides either physically or with vlans so you can keep the traffic separate for monitoring, scaling etc.

 

The various roles you have there are largely fine for virtualising and actually some of your existing tin isn't bad so you could quite easily find yourself a physical DC and potentially a backup server attached to the LTO library for D2D2T if you're playing with DPM. (I'm also a stickler for making sure you have some form of 'out of the server room' back up and if you have a capable LTO5 library I'd go for it).

 

I'd break the printserver onto it's own box and let your DCs be DCs too, I'm not a fan of mixing roles and in a virtual environment there's not normally a need to!

 

Virtualising the rest should be relatively painless but you'll want to make sure you specify enough spindle to handle the fileserver requirement (or everything else will suffer) as I'm a bit uneasy about the number of them + their disks! ( How much data is being stored on them? )

 

I think you'll have a fun project on your hands with this but you should be fine to go at it with a couple/3 HP DL360's stuffed with RAM (go for big modules even if you don't fill the board so you can expand later), a decent SAN and give your network a good think through and I think you'll be laughing.

 

Oh, and I don't think you need to worry about 10GbE if I'm brutally honest in your budget at the moment (far more interesting things to buy), I'd probably just look at quad port gig cards in your boxes and team them together but if you do look at replacing switches it's definitely something you need to think about so look for decent switching throughput/fabric and the expansion ports being decent. HP again not a worry.

 

Sorry, not many specifics in there just more thought provoking stuff.

[MrWu]

Thanks kmount;

 

What a fantastic community this is, I hope once things settle I could help others too.

 

I think speaking to the ex Network Manager the idea was to spread the Data by year groups for a bit of load balancing and redundancy..which did make sense. I do have concerns having one big file server but my DL380 Filesever from my old company ran without a hitch for over 4 years..however being a school enviroment the logon patterns are totally different from a commercial office I suppose

 

Absolutely agree on having a Physical DC outside of Hyper-V, I've heard horror story from my IT friend trying to sort out a client onsite that had a SAN issue and all the DCs were virtualised...

I would probably keep the fileserver as it's own phyiscal box as well, let it handle just data duties (with at least a teamed NIC)....I might get a high performance switch to link the servers together.

 

Thank you and welcome any more suggestions ! :-)

[kmount]

I just noticed you're newish here, welcome You've been here a month and not ran away screaming for the hills, that's a good thing.

 

I am sure you do & will continue to contribute just fine!

 

You might be able to run the fileserver off of the QSAN box if it's any good (I've never used them), and back it up across onto whatever new toy comes along to play with.

 

That said, a DL380 with a bunch of disk will probably work just as well (decent disks!!) as a fileserver and you could start your virtualisation project on top of the QSAN but it depends how resilient it is as a system as it's one of the great unknowns!

 

Of course, this is even assuming you NEED a SAN at the moment, you might be able to get away with (as many folks do) a few boxes each with some local disks raided up and virtualise on top of that; yes you'll lose the vmotion and ability to shift stuff around as easily but you might not need that if you have a decent raid set up and back up regime.

 

You've made (and I'm pleased, so many people don't) the right connection that comparing the logon/logoff patterns in commercial to Edu simply isn't close. You need kit that can take almost the full complement of your student PC estate logging in & out every hour in large chunks so a/some capable fileserver and sensible profile management is the biggest win you could possibly get. There's probably some argument here also that whatever you do do in fileserver land if you only deploy one for now it's probably still worth setting it up in DFS style so if you need to bolt on more fileservers later you don't need to worry about updating unc path links and messing about you just throw the box into the mix.

 

What's your network like at the moment? Do you have a reasonable core switch that the servers are connected close to or is it a bit of a mish mash?

 

So many things to think about

[glennda]

This Could be quite long! As other have mentioned server wise everything is HP (apart from my custom built backup server) Our Lan Infrastructure is all HP Procurve (2610's as edge switches with a 5406Zl core switch providing the routing)

 

I was in a similar position to you about 2 1/2 years ago with what you have now. This is what I would do (and in most cases have just with older kit DL380 G5's)

 

Virtual hosts - x 3/4

HP DL360 G7's Dual 6 core Xeon with 48GB ram in each (that model comes with 12Gb but easily upgraded - supports 384Gb) this machine has 4 Nics so additional cards would be needed if you get a new ISCSI SAN. If running windows for Hyper-V 2x72GB drives for OS. If ESXI an SD card (server has slot directly on motherboard).

To Host:

• File Servers (however many you like 4/6Gb ram each)
  
• Exchange Server x 1 (Server 2008 R2 16GB ram)
  
• RDP Server x 1 (Server 2k8 R2 10Gb Ram
  
• Sims Server x 1 (Server 2008 R2 Full Sql 8Gb ram - if not more)
  
• Intranet Webserver x 1 (I would say apache on linux)
  
• Impero Server x 1 (not sure hardware requirements)
  
• 1 X Frog Server (not sure hardware requirements)
  
• Dedicated Print Server (Server 2003/8 R2 depending on clients - 4/6GB Ram should do)
  

SAN

This is where I am not sure and possibly others can comment better. I use a Fiber Channel San but can be expensive. Although I can get 8GB per channel to the device. I use an HP MSA2012FC with additional enclosure (about 8TB raided). HP currently have an Offer on the HP P2000 G3 array with up to 50% cashback.

 

Physical DC x 1

HP Dl360 G7 Quad Core Xeon with 6GB ram. Should be plenty 2 x 250Gb SAS Raid 1 - This can host DNS, DHCP, WSUS (Server 2008 R2)

 

As for backups I would go with a fairly decent machine connected to your existing ISCSI San (preferable in a separate location is possible) I can keep 7 weeks of backups on 16TB so should be fairly good. Can also use Nas device aswell (if it supports ISCSI good - if not USB/Firewire?) I've never used DPM so not sure what it's like hardware wise but the specs are here.

 

For each Virtual host Look to be paying around 4k (from what i ordered and got this should be doable. I got mine from BT Business Direct with 24Gb ram in each and extra 4port nic card for 3k ish (then money was claimed back from HP for Ram upgrade). Then the Physical DC and backup Server about £1500-£2000. In total 16-20k for servers and the rest on a San.

Edited January 7, 2012 by glennda

[MrWu]

I just noticed you're newish here, welcome You've been here a month and not ran away screaming for the hills, that's a good thing.

 

I am sure you do & will continue to contribute just fine!

 

You might be able to run the fileserver off of the QSAN box if it's any good (I've never used them), and back it up across onto whatever new toy comes along to play with.

 

That said, a DL380 with a bunch of disk will probably work just as well (decent disks!!) as a fileserver and you could start your virtualisation project on top of the QSAN but it depends how resilient it is as a system as it's one of the great unknowns!

 

Of course, this is even assuming you NEED a SAN at the moment, you might be able to get away with (as many folks do) a few boxes each with some local disks raided up and virtualise on top of that; yes you'll lose the vmotion and ability to shift stuff around as easily but you might not need that if you have a decent raid set up and back up regime.

 

You've made (and I'm pleased, so many people don't) the right connection that comparing the logon/logoff patterns in commercial to Edu simply isn't close. You need kit that can take almost the full complement of your student PC estate logging in & out every hour in large chunks so a/some capable fileserver and sensible profile management is the biggest win you could possibly get. There's probably some argument here also that whatever you do do in fileserver land if you only deploy one for now it's probably still worth setting it up in DFS style so if you need to bolt on more fileservers later you don't need to worry about updating unc path links and messing about you just throw the box into the mix.

 

What's your network like at the moment? Do you have a reasonable core switch that the servers are connected close to or is it a bit of a mish mash?

 

So many things to think about

 

Thanks again :-) Believe me there were times I would have flung myself down the hill curled up in a ball...it's just this week made me stronger, with all the powercut issues, still during school time the head master managed to host his assembly, no one lost data, and all classes managed to log on ok without even knowing the night before they had no fileserver ! That made me keep going.

 

It's a huge Mish Mash at the moment, I'm hoping it's untidy more then anything else (it's all CAT5e / a bit of CAT 6 horizontal) with fibre between cabinets. The server room has a Dlink 24port gigabit switch and a mixture of fibre converters (dangling down like Xmas decorations !) some are even 100mbps only I think. We spent 3 days tidying up the ICT office, storeroom and the front of the server room, so will need to visit all the comms cabinets next term break if not earlier ! I'm hoping 15k is more than generous to sort that lot out and mostly be elbow grease..

[MrWu]

This Could be quite long! As other have mentioned server wise everything is HP (apart from my custom built backup server) Our Lan Infrastructure is all HP Procurve (2610's as edge switches with a 5406Zl core switch providing the routing)

 

I was in a similar position to you about 2 1/2 years ago with what you have now. This is what I would do (and in most cases have just with older kit DL380 G5's)

 

Virtual hosts - x 3/4

HP DL360 G7's Dual 6 core Xeon with 48GB ram in each (that model comes with 12Gb but easily upgraded - supports 384Gb) this machine has 4 Nics so additional cards would be needed if you get a new ISCSI SAN. If running windows for Hyper-V 2x72GB drives for OS. If ESXI an SD card (server has slot directly on motherboard).

To Host:

• File Servers (however many you like 4/6Gb ram each)
  
• Exchange Server x 1 (Server 2008 R2 16GB ram)
  
• RDP Server x 1 (Server 2k8 R2 10Gb Ram
  
• Sims Server x 1 (Server 2008 R2 Full Sql 8Gb ram - if not more)
  
• Intranet Webserver x 1 (I would say apache on linux)
  
• Impero Server x 1 (not sure hardware requirements)
  
• 1 X Frog Server (not sure hardware requirements)
  
• Dedicated Print Server (Server 2003/8 R2 depending on clients - 4/6GB Ram should do)
  

SAN

This is where I am not sure and possibly others can comment better. I use a Fiber Channel San but can be expensive. Although I can get 8GB per channel to the device. I use an HP MSA2012FC with additional enclosure (about 8TB raided). HP currently have an Offer on the HP P2000 G3 array with up to 50% cashback.

 

Physical DC x 1

HP Dl360 G7 Quad Core Xeon with 6GB ram. Should be plenty 2 x 250Gb SAS Raid 1 - This can host DNS, DHCP, WSUS (Server 2008 R2)

 

As for backups I would go with a fairly decent machine connected to your existing ISCSI San (preferable in a separate location is possible) I can keep 7 weeks of backups on 16TB so should be fairly good. Can also use Nas device aswell (if it supports ISCSI good - if not USB/Firewire?) I've never used DPM so not sure what it's like hardware wise but the specs are here.

 

For each Virtual host Look to be paying around 4k (from what i ordered and got this should be doable. I got mine from BT Business Direct with 24Gb ram in each and extra 4port nic card for 3k ish (then money was claimed back from HP for Ram upgrade). Then the Physical DC and backup Server about £1500-£2000. In total 16-20k for servers and the rest on a San.

 

Cheers for the fantastic info, have printed this out and go through with a fine toothcomb ! :-)

[glennda]

Cheers for the fantastic info, have printed this out and go through with a fine toothcomb ! :-)

 

Not a Problem - and Questions either post them up here or PM me. As I say I have done most of the above anyway although I don't use either Hyper-V or ESXI as i host on Linux KVM.

 

For Backups if you don't want to use your ISCSI San or want to use it for the VM's (although not sure of its spec/up to the job) take a look at this thread Thats what I made in the end - 32Tb backup server!

[glennda]

If you go ISCSI make sure you have a backbone switch that can cope. For the Fiber Channel stuff we use an HP 20/8Q to distribute between the multiple hosts.

[kmount]

If you go iSCSI shove it on a separate switch, no need to make your core cry unnecessarily unless you need to get at it from elsewhere in the building (but even then you could cheat this).

 

Excellent post from glennda, rep+

[glennda]

Also with regards to UPS's - a quick flick through the APC site and loading the above in (with an MSA P2000 San as the example) and a core switch (so the UPS can communicate with servers when power goes - spend ages working out once why my servers didn't shut down - core wasn't on ups! ) It reckons something with about 6KVA with an additional battery pack should give you 45 mins power (which is pretty good!) The APC one is SURT6000XLI + SURT192XLBP which comes to about 3k. I Personally have the 10KVA one which runs my entire server rack from a 32A Three phase connector - I will be buying an additional one next year to load balance. You will probably need a PDU as well. Something like this APC one

[MrWu]

Thanks guys;

 

I have also had a brainwave..

 

A HP DL380 G7 was ordered for the new finance software (We are going into Academy status - I didnt include it in my plan because it was already ordered..but solution was staring at my face lol) the spec is:

 

HP ProLiant DL380 G7 - Intel Xeon E5620 / 2.4 GHz ( Quad-Core ) 2U Server / 6Gb Installed / DVD R/W Optical Drive / 3 Years - On-Site NBD / (Part Number - 470065-479)

2 GB RAM upgrade

8 x507127-B21 (300GB 10k Drives)

Redundant PSU

 

The Bursa wants it as a finance server only but I think I can convince him I can virtualise this, I could possibly get another CPU and some more RAM and and virtualise the Finace software (it will run on SQL 2008, requirements not that heavy:

 

Data Server

 

Software

Microsoft SQL Server 2005 Standard or above

Processor: Dual 1.5 GHz or better

Operating System:Microsoft Windows 2003 Server or above

Memory: 2 GB

Disk : 10GB

 

 Clients running Novel 4 or above can use Sybase SQL Anywhere V7 or V8

 Clients running UNIX can operate with Oracle 8i or above

 If running Windows 2003 or 2008 and SQL 2005 or 2008 please increase memory by a minimum of 2Gb

 It is the client’s responsibility to ensure the hardware is capable of running these database platforms

 

Web Server

Software

All supplied by Company(i.e. IIS is not required)

Processor : Intel Pentium Processor, 1.5GHz or better

Operating System: Microsoft Windows 2003 Server or above

Memory:2 GB

Disk : 10GB

 

 Clients who wish to may operate the data server and web server from the same physical machine. In this case we would recommend increasing the memory and a better processor

 

So I could put the 8 x 300GB Disk on RAID 10 on this new G7, add more RAM and hopefully and 2nd Xeon, would this be good to go for as a Hyper V and maybe another 2 virtual machines (Impero, Intranet Servers)? For guys who have ran SQL on Hyper-V, does it really need passthrough disk/ Spindle as Microsft recommends?

Edited January 8, 2012 by MrWu

[glennda]

For anything that runs using sql the more ram it can have the better. I run Sims with 16gb ram so that the sql process can store as much data in the ram as possible therefore lowering the load on the disks (our sims+fms db is aroun 9gb combined)

[MrWu]

For anything that runs using sql the more ram it can have the better. I run Sims with 16gb ram so that the sql process can store as much data in the ram as possible therefore lowering the load on the disks (our sims+fms db is aroun 9gb combined)

 

Thanks glennda;

 

I think if the new finance software is succesfully virtualised, then SIMS will go along this route as well.

So I'm thinking of just running the Hyper-V Visor on the Host machine. RAID 10 for the 8 x 300GB Disk (should give 1.2 TB of space)

Do you use passthrough disks or a fixed VHD for your sims + fms?

 

My only concern is even with 1.2TB it might not be futureproof enough to host say 3-4 machines.

 

Depending on how critical uptime is (and costs) I could use a fast SAN as suggested and have Fallover protection with another DL380 G7.

 

We are going to implement MDT/DPM 2007/2012 as well so server rebuilds should be quite quick and straightforward, so medium term that might be the option.

[glennda]

I use the equivalent of VHD's for my all my servers but as I don't use/have never used Hyper-V you may need to ask others.