Serverless Schools - Thoughts, Tips and the Reality!

[localzuk]

All servers are centralised, just central to what, the room, the building, the city, the country, the world?

Where you put the server depends on the users, if they're all over the world, put your server in the world, if they're just on one site, put the server on the site.

What we do need is different words for the computer and the software. Can we just say computer instead of physical server machine?

"I'm renting some computers from Google" vs "I bought 3 computers to run servers on"

Sorry but this is just semantics. With a Trust, centralised means exactly what it means. The other schools have no servers, only the central server room does. Centralised.

Also, the words used are clear from context. "Centralised servers", in context, means the equipment is all hosted in a central site.

My server says it's using 80 Watts, that's 700 kWh, so £100 in power per year

Our cluster runs servers with dual 700W PSUs, 4 of them. Those are standard rackmount servers. Nothing special. A single server running an 80W PSU is most certainly not going to provide services covering 30 schools!

Edited March 13, 2019 by localzuk

[supportman]

By throwing away the stuff that works and doesn't cost me anything and only working on things I have to pay for monthly?

 

Well yep that's progress for you I'll admit to really enjoying not having to worry about local servers so much these days and happy to pay for that privilege.

 

Saying that, Microsoft have hugely increased their local server pricing in recent years. I don't think its any more expensive to "go cloud" it these modern times.

[Achandler]

Until they bump up the cloud pricing when they have captured enough of the market.

[mavhc]

Our cluster runs servers with dual 700W PSUs, 4 of them. Those are standard rackmount servers. Nothing special. A single server running an 80W PSU is most certainly not going to provide services covering 30 schools!

 

80W is what it's actually using, not what the PSU is rated to. As I read it the calculations were for 30 servers, one in each school.

[Mr.Ben]

Sorry but this is just semantics. With a Trust, centralised means exactly what it means. The other schools have no servers, only the central server room does. Centralised.

Also, the words used are clear from context. "Centralised servers", in context, means the equipment is all hosted in a central site.

Our cluster runs servers with dual 700W PSUs, 4 of them. Those are standard rackmount servers. Nothing special. A single server running an 80W PSU is most certainly not going to provide services covering 30 schools!

 

^This

 

My calculations don't assume we are using full load with dual 450W supplies:

 

 

Total Wattage across power supplies 900

Estimated Average Load 400

 

kWh Day Cost Yearly Cost

Energy Consumed Per Day 9.6 N/A N/A

Energy Consumed during Day Rate 6.8 £1.11 £403.57

Energy Consumed during Night Rate 2.8 £0.33 £121.52

 

Total Yearly Cost

£525.09

 

That is per server, ignoring the cost of running storage as well.

[mavhc]

Where does the 400W figure come from though? Is this a quad cpu machine running 20 HDDs?

[Mr.Ben]

This is what I'm working on:

 

2x Intel Xeon Processors (Average power Load based on complex workload) - 170W for the pair

 

https://www.intel.com/content/www/us/en/products/processors/xeon/scalable/silver-processors/silver-4112.html

 

Motheboard - 100W

 

RAM (16GB DDR4 @ 2133) - 6W

 

https://www.tomshardware.com/reviews/intel-core-i7-5960x-haswell-e-cpu,3918-13.html

 

6x HDD (2+3+1) 20W

 

DVD drive (Idle) 2W

 

6x 2000 RPM fans = 24W

 

+ 32W inefficiency for dual supplies

 

Total is 354W - I have rounded up as the kit schools currently have servers at least 5 years old.

[mavhc]

85W per cpu is max though, who has cpus that are running at 100% 24/7? they're usually nearer 0%. 100W for a motherboard? I'd just measure the actual power usage, it'll be way lower. Handily iLO etc can do that for you remotely

[localzuk]

85W per cpu is max though, who has cpus that are running at 100% 24/7? they're usually nearer 0%. 100W for a motherboard? I'd just measure the actual power usage, it'll be way lower. Handily iLO etc can do that for you remotely

My cluster idles at 40% CPU usage (as it is running >40 VMs). Active usage can see it going up to 80% usage during the day.

[mavhc]

My cluster idles at 40% CPU usage (as it is running >40 VMs). Active usage can see it going up to 80% usage during the day.

 

What are the 40 VMs for, how many users? How many cores? What's using 40% when it's idle?

[DrBeaker]

My cluster idles at 40% CPU usage (as it is running >40 VMs). Active usage can see it going up to 80% usage during the day.

 

40x VMs? What are you running?

[chaplic]

Very Interested in this thread, Our ovs-es expires in june. Small school with 5FTEs

 

We have two "servers" (PCs with RAID controllers" running 2012R2 Domain Controllers and DFS. O365 for email has been a great hit for us so I want to expand on that and go serverless

So

 

File Servers--> Sharepoint

Personal Drive-> Onedrive

PC policy settings-> Intune

Domain Controller Login -> Azure AD Domain Joined machines

PC Build -> not sure, probably MDT build on a usb stick

Patch Management->Assume can do thru intune, haven't checked that out

On prem kit audit (spiceworks) -> Not sure, will probably do without, or run on a PC that will also be used for testing etc.

DHCP-> ???

Ubiquiti Controller -> ?? maybe the PC as above

 

Questions or ideas welcome

[free780]

Using Azure AD instead of ADFS/LDAP works for quite a lot of apps but they need to support SAML/WIA. I don't know about spice works you also need to run it somewhere which will incur cost. You'll need dhcp from somewhere even if it's a switch. OU can do windows updates via intune but not 3rd party patching.

[chaplic]

I was pondering a couple of Rasperberry PIs for DHCP but had forgotten about the capability of our switches which will do DHCP, and I have two of those in different locations so thanks for that :-)

 

Using Azure AD Enterprise apps and SAML and all that fun is my day job, but given there was no AD when I rocked up there 3+ years ago I am pretty confident no issues here. I've concluded that a server of sorts is needed for PRTG and Spiceworks, but this could be a laptop hidden in the comms cabinet running windows 10 and has the advantage of a built in UPS.

 

 

Actually, that's another thought - currentlly I use Domain Based KMS (so not a KMS server but using AD) - I am not aware of a replacement of that?

[free780]

How about subscription licensing for Windows 10 via intune? Office 365 Proplus is cloud licensing anyway. So unless you have any 3rd party software that needs a licensing server.

[Mr.Ben]

Very Interested in this thread, Our ovs-es expires in june. Small school with 5FTEs

 

We have two "servers" (PCs with RAID controllers" running 2012R2 Domain Controllers and DFS. O365 for email has been a great hit for us so I want to expand on that and go serverless

So

 

File Servers--> Sharepoint

Personal Drive-> Onedrive

PC policy settings-> Intune

Domain Controller Login -> Azure AD Domain Joined machines

PC Build -> not sure, probably MDT build on a usb stick

Patch Management->Assume can do thru intune, haven't checked that out

On prem kit audit (spiceworks) -> Not sure, will probably do without, or run on a PC that will also be used for testing etc.

DHCP-> ???

Ubiquiti Controller -> ?? maybe the PC as above

 

Questions or ideas welcome

DHCP can be provisioned on most routers.

 

DNS can be pointed directly at your ISP.

 

Intune does an ok job at auditing.

 

Patch management is through Intune, but it is more rollups and SAC based.

 

Buy a Cloud key for Ubiquiti - it's about £60 and runs the management software.

[Mr.Ben]

I was pondering a couple of Rasperberry PIs for DHCP but had forgotten about the capability of our switches which will do DHCP, and I have two of those in different locations so thanks for that :-)

 

Using Azure AD Enterprise apps and SAML and all that fun is my day job, but given there was no AD when I rocked up there 3+ years ago I am pretty confident no issues here. I've concluded that a server of sorts is needed for PRTG and Spiceworks, but this could be a laptop hidden in the comms cabinet running windows 10 and has the advantage of a built in UPS.

 

 

Actually, that's another thought - currentlly I use Domain Based KMS (so not a KMS server but using AD) - I am not aware of a replacement of that?

So far I have used MAK keys - Microsoft ave been very accomodating in raising my limit in my VLSC OVS portal. Apparently it can go up to 7,500.

[mavhc]

DHCP can be provisioned on most routers.

 

DNS can be pointed directly at your ISP.

 

Intune does an ok job at auditing.

 

Patch management is through Intune, but it is more rollups and SAC based.

 

Buy a Cloud key for Ubiquiti - it's about £60 and runs the management software.

 

ISP DNS won't work for internal DNS though.

 

Isn't Cloud key a server? You've broken the rules!

 

We're serverless, except for all these servers we have. Everything is a computer, everything is a server.

[Mr.Ben]

You are technically correct, but you are arguing on semantics. Perhaps this thread should be entitled: How to use cloud services effectively to reduce capital outlay and move to an operating model of funding whilst ensuring minimum disruption to our schools primary goal of education. Catchy.

 

Why would you need internal DNS if all of your resources and tools are in the cloud?

 

At worst, I'll be spinning up an internal DNS in a centralised location, but with the current setup I'm yet to be convinced that it is required, and would be another technical resource to maintain - we don't need any split brain DNS and nothing that we can afford from a primary school perspective requires any specific records.

 

The cloud key's functions could be spun up in Azure, but that is more costly for a single school than the £60 outlay.

[HPlum78]

Green field site = cloud. The business would have to make a very strong case for building data center on site.

 

Central server room hub/ spoke setup, i would argue that you have built a cloud and would guarantee that you will not be doing it as cheap as MS/ AWS. (nor have you taken into consideration the redundancy that a cloud can offer, I would guess you are not going to come close to)

 

I am unsure of the workings out of the actual cost of running a sever on site... I see no costs for cooling/ maintenance for all equipment involved and disposal, this is the short list! (all of this makes it a little more than £100-150 a year)

 

I have said this more than once, for those who are not in a position to migrate to the cloud. Its low hanging fruit, you are not going to save a penny if you are consuming IaaS that's just not smart. Also its no use mixing and matching so if an application is going to have a dependency on something on prem then it's on prem until you can bite the bullet and cloudify an entire solution.

 

SaaS solutions are where the biggest savings are, PaaS after that. An SQL PaaS solution in azure works out cheaper than building an on prem solution for the average workloads. An IIS Web site/application can be run for pennies if the right plan is selected.

 

Azure functions costs are less than that as MS use the cycles that those people running VMs don't make use of (don't tell them mind!) something that you cannot do in you centralised data centre! (unless you have built your own service bus)

 

No on prem hardware is a long way off for most, selective cloud is here and we should use it where it makes the most sense.

Edited March 16, 2019 by HPlum78

[mavhc]

But what school software doesn't rely on some dodgy installation of MSSQL version 3 versions ago?

 

Exam software is all terrible for a start.

 

Buy a cheap server, put a few TB SSDs in, 4x 1Gb connections, will file serve 1000 clients easily for £100 a year in electricity and let's say £100 a year in cooling, vs what, £8000/year per Gb internet connection, need 2 of course for redundancy, and now it's still only half the speed

[Mr.Ben]

I am in a unique position being all primary, but if a single site required an ancient piece of software to support a £3k server I would still go cloud and replace the software - the school have two choices, pay for the extra time and knowledge to support the package or replace it - replacing it will be cheaper than hosting.

 

If you retained that piece of software on site, bringing everything else cloud based, then the school would have to consider the costs of staffing that equipment, especially when it goes wrong, or investing in kit that camn be remotely managed (using iDrac and such)

 

The geographic distance between my sites ensures that I have to consider operational and technical efficiency. If I pay a tech £22k (with a cost of £28.6k to the trust when pensions et are taken into account), with a working pattern of 48 weeks his per hour cost is £15.90 per hour. Assume that 20% of the technician's time is wasted and the cost rises to £19.90 an hour.

 

The average distance between our offices and sites is 1 hour. Assume an hour on site and a distance each way of 30 miles and I've spent £87 on a single visit, excluding any consideration to the risks of HR and staffing.

 

I could run a low end Azure VM for £59 per month that can service all 30 schools. (We do this for our finance system).

 

My core drive it is about TCO, whilst delivering modern technology - cloud does make sense in my circumstances.

 

As for the Gb connection, what are peoples experiences?. In the two cloud schools I'm running I'm seeing less than 50Mbps at peak times, apart from the few days that we imaged all of the machines and downloaded lots of software (when I topped out the 80Mbps). In my previous role at a local auithority, we backhauled traffic for 130 sites through a 1Gbps connection. The only time in the past 5 years that we topped it out (our average was 300Mbps) was during Euro 2016 when England played Wales.

 

 

With the new connections I'm looking at for our schools (which come in at around 4.5k for leased lines (100/100) and 3k for FTTC (80/20) , peering is more important to me - the network we connect to must peer directly with MS, Google and Apple as that keep traffic off of the public internet.

Edited March 16, 2019 by Mr.Ben

[mavhc]

So do your techs never travel to the schools, and you do everything remotely, or would they already be there because they do travel to the schools anyway? The only reason to pay £3k for a server is so you can have dual psus and remote lights out management in the first place, otherwise might as well use a random desktop computer for £300.

 

99% of files are accessed from on site, so that's where my file server is, simple really, 30 pupils editing video over an internet connection would be a bit weird. That's without the rest of the school trying to use the same connection. Got solar panels so it costs nothing to run.

[Mr.Ben]

We don't have any techs at the moment, I'm starting from scratch, but I am trying to keep the staffing levels to a minimum.

 

Currently our schools pay around £130k across the trust for support from local companies. If I was to bring this in house I would envisage 2x Field Techs and a Helpdesk member of staff at most. (less than £90k). There are other considerations - for £22k I'm not going to be getting the most knowledgeable staff so will have to train them, ensure sickness cover and bear the other costs of their employment so it is really important that we are as efficient as possible.

 

Field techs would likely be road warriors, working from specific sites on a daily basis to ensure that schools see a face from the IT department, however I do not want the techs to be reacting and travelling at the drop of hat between sites - that costs more money (work on £50 per site visit at a minimum) - I need their time planned for projects, updates, rollouts and training our staff, not chasing round after failed HDD's and the replacement/restoration of local servers.

 

There are a lot of politics that you may not be considering in large organisations. It is crap but you have to deal with it - schools only see the numbers and expect a quality of service way above what they can afford, even more so when they don't invest in IT and that is to the detriment of us all, especially those we are educating.

 

 

I'm happy to let schools sweat their client assets, but not servers as this does correlate with more expensive technical time. Cloudless takes the decision away from schools, potentially reduces costs and gives the Trust more ability to offer compliance with our legal requirements (for which we are ultimately responsible)

 

I want to build a proactive, managed service that has projects that can be reliably delivered, without consistent firefighting - it's either lights out servers or a cloud based approach. Bear in mind a single project can take a person 3 months at a time to roll out.

 

Managing the servers On-Prem would also require a higher quality member of staff (and more cost to get them in addition to the hardware costs).

 

With the exception of 2 Azure VM's (shared between all schools) the cloud based trial schools rely on no windows servers. Files are in Sharepoint and OneDrive, at no additional cost beyond that we pay for our OVS-ES.

 

VLAN based QoS for Staff/Admin devices or application aware Class of Service would sort the issue of prioritising of web traffic.

 

Solar is cool - we have a couple of sites with panels, but the total cost of install now is so high we wouldn't be able to afford it across the board.

Edited March 16, 2019 by Mr.Ben

[supportman]

We don't have any techs at the moment, I'm starting from scratch, but I am trying to keep the staffing levels to a minimum.

 

Currently our schools pay around £130k across the trust for support from local companies. If I was to bring this in house I would envisage 2x Field Techs and a Helpdesk member of staff at most. (less than £90k). There are other considerations - for £22k I'm not going to be getting the most knowledgeable staff so will have to train them, ensure sickness cover and bear the other costs of their employment so it is really important that we are as efficient as possible.

 

Field techs would likely be road warriors, working from specific sites on a daily basis to ensure that schools see a face from the IT department, however I do not want the techs to be reacting and travelling at the drop of hat between sites - that costs more money (work on £50 per site visit at a minimum) - I need their time planned for projects, updates, rollouts and training our staff, not chasing round after failed HDD's and the replacement/restoration of local servers.

 

There are a lot of politics that you may not be considering in large organisations. It is crap but you have to deal with it - schools only see the numbers and expect a quality of service way above what they can afford, even more so when they don't invest in IT and that is to the detriment of us all, especially those we are educating.

 

 

I'm happy to let schools sweat their client assets, but not servers as this does correlate with more expensive technical time. Cloudless takes the decision away from schools, potentially reduces costs and gives the Trust more ability to offer compliance with our legal requirements (for which we are ultimately responsible)

 

I want to build a proactive, managed service that has projects that can be reliably delivered, without consistent firefighting - it's either lights out servers or a cloud based approach. Bear in mind a single project can take a person 3 months at a time to roll out.

 

Managing the servers On-Prem would also require a higher quality member of staff (and more cost to get them in addition to the hardware costs).

 

With the exception of 2 Azure VM's (shared between all schools) the cloud based trial schools rely on no windows servers. Files are in Sharepoint and OneDrive, at no additional cost beyond that we pay for our OVS-ES.

 

VLAN based QoS for Staff/Admin devices or application aware Class of Service would sort the issue of prioritising of web traffic.

 

Solar is cool - we have a couple of sites with panels, but the total cost of install now is so high we wouldn't be able to afford it across the board.

 

Sounds like a great, modern plan. Keep us updated how it goes as this is surely the future.