Single disk server?

[sparkeh]

55 mins to wait to access restored data due a failed disk is, in my opinion, 55 mins too long.

 

Remember, under GDPR, availability will be a factor. Prolonged unavailability will be classed as a breach.

[matt40k]

Quite an interesting topic - years ago you had only 1 option in terms of storage, spinning rust. Let's face it anything with moving parts in a server will go wrong long before the non-moving parts. HDD being really sensitive - to shock and

- and it makes sense when you think they are spinning at 7,200 or even 15,000 RPM and the platters are separated by such a small space. So we got around this by having 2 (or more). So do we still need RAID for SSD? Well SSD (assuming we're talking server SSD that are designed for the load and not cheap SSD) are more reliable then 2 (or more) traditional hdds, in terms of SSD vs raid traditional HDD, SSD wins hands down. So yer, it's a valid question, specially when you think two SSD installed at the same time, writing the same data is prob going to die at the same time.

 

The other interesting points are GDPR - eh? Backups of critical data that will land you in trouble will be need to be in two completely different media formats. Personally - paper and electronic - if it has to be electronic for both, one should be CSV. Remember those random posts we get here asking about can someone help get data out of that really old database, but they don't have a copy of the old client software and waste weeks and weeks trying to get the data. Doesn't matter that you have 12 copies of that file or disk\tape\cd\cloud\whatever, if you can't read it, it's not a backup. If you've got printed data collection sheet in case of power loss\IT failure\whatever, is this still an issue??

 

The final interesting one was - servers are cattle, not pets. Sounds like alot of folks still have pets. So Microsoft or Google gives you free email, makes complete sense, no one should have on prem exchange, doesn't make sense. They also give your users 1TB of space. So you don't need alot of space on prem, the only things you should be worried about are legacy things - most likely your MIS. So if the bulk of your stuff is now in the in other peoples server rooms, why don't you spin up vms in their server room? Yes AWS \ Azure looks expensive, but you don't use it, you don't pay. You separate out your storage from your compute, storage you can't escape, but it's pennies, compute you only pay for when you need it - both Azure and AWS has ways of start\shutdown VMs etc. So you looking at what 8am - 4pm, 5 days a week then take off all the school holidays - fine, you'll have some edge cases that need a few more hours.

 

So I guess my question is, who cares? And why do you care about the cattle?

[3s-gtech]

We've had little/no internet access this week. Going cloud for everything? No thanks.

[Arthur]

specially when you think two SSD installed at the same time, writing the same data is prob going to die at the same time.

Interestingly with software-based RAID1 in Linux, the first SSD will receive a disproportionate number of writes compared to the second SSD so will probably die first.

 

https://utcc.utoronto.ca/~cks/space/blog/linux/UnbalancedSSDMirrorReads

 

It turns out that this is more or less expected behavior in Linux software RAID, especially on SSDs, and has been for a while. It appears that the core change for this was made to the software RAID code in 2012, and then an important related change was made in late 2016 (and may not be in long-term distribution kernels).

 

The current state of RAID1 read balancing is kind of complex, but the important thing here in all kernels since 2012 is that if you have SSDs and at least one disk is idle, the first idle disk will be chosen. In general the read balancing code will use the (first) disk with the least pending IO, so the case of idle disks is just the limit case.

 

(In kernels with the late 2016 change, this widens to if at least one disk is idle, the first idle disk will be chosen, even if all mirrors are HDs).

 

SSDs are very fast in general and they have no seek delays for non-sequential IO. The result is that under casual read loads, most of the time both SSDs in a mirror are idle and so the RAID1 read balancing code will always choose to read from the first SSD. Reads spill over to the second SSD only if the first SSD is already handling a read at the time that an unrelated second read comes in. As we can see here, that doesn't happen all that frequently.

[matt40k]

Seriously?!? Has no local entrepreneur started laying their own fibre? Or some sort of wireless solution? Are they happy being in the dark ages where you are?

[DJ-1701]

Seriously?!? Has no local entrepreneur started laying their own fibre? Or some sort of wireless solution? Are they happy being in the dark ages where you are?

 

A money making opportunity for you @matt40k.

Right, hands up those with slow rural connections first.

[3s-gtech]

Seriously?!? Has no local entrepreneur started laying their own fibre? Or some sort of wireless solution? Are they happy being in the dark ages where you are?

 

We live in a colossal county with the total population of a large town. In Wales. The road network is still a series of old tracks connected by small trunk roads - the broadband and fibre network is comparable. It's LEA leased fibre or nowt. When it works, it's pretty good. But I don't feel we're in the dark ages - we provide all of the features that many 'cutting edge' schools do. We just host them ourselves (other than our supplemental O365 tenancy without mail).

[Norphy]

.

 

So I guess my question is, who cares? And why do you care about the cattle?

 

Because farmers who don’t care about their cattle end up with no cattle. You might have the same level of emotional attachment to livestock as you would with a pet but you can be damn sure that whoever keeps a cow cares about it.

[matt40k]

@DJ-1701 I live near Adastral Park, there are plenty of more talent folks around here that could provide a far better solution then I ever could.

@Norphy You're missing the point, schools aren't in the business of cattle. AWS \ Microsoft \ etc ARE in the business of cattle. As a consumer I care about the end product, the cattle is a means to an end - food. The food isn't my business, its a resource I require to do my business. Bottom line, I don't care about the cattle so long as it gives me the resources I require. If I can buy the bacon at the supermarket, why am I keeping pigs? The economics of scale means having your own cattle doesn't make sense.

 

The internet is such a major innovation we shouldn't be doing workarounds, we should be fixing these issue - ideally whilst we can see get some of that EU money!!

[zag]

SSD's can fail just as much as a mechanical HDD.

 

This simply isn't true from our experience, SSDs are a magnitude more reliable than mechanical spinning disks.

 

If you have experienced as many SSD failures as hard disks, then you are simply buying the wrong brands.

[mavhc]

@DJ-1701 I live near Adastral Park, there are plenty of more talent folks around here that could provide a far better solution then I ever could.

@Norphy You're missing the point, schools aren't in the business of cattle. AWS \ Microsoft \ etc ARE in the business of cattle. As a consumer I care about the end product, the cattle is a means to an end - food. The food isn't my business, its a resource I require to do my business. Bottom line, I don't care about the cattle so long as it gives me the resources I require. If I can buy the bacon at the supermarket, why am I keeping pigs? The economics of scale means having your own cattle doesn't make sense.

 

The internet is such a major innovation we shouldn't be doing workarounds, we should be fixing these issue - ideally whilst we can see get some of that EU money!!

 

What's the issue though? Renting a server costs the same as buying it, and then you have to pay extra for better internet access. If your school is highly distributed, everyone's learning from home or similar, that's fine, but everyone's in 1 building with 1/10Gb server connections in most cases.

 

Leave internet things like email on the internet, email's pretty useless if you don't have internet access anyway.

 

If you backup to the internet then by all means have the ability to restore to a VM in the same datacentre in seconds. Or sync your LDAP with an internet LDAP.

 

A hybrid solution is great, but uploading everything over a 1Gb link only to download it again seems weird.

[zag]

I was surprised to read you had 8 physical servers, seems a lot.

 

How do you use them? How many SSDs per server? How much user storage? Are backups also SSD? Do you keep long term backups?

 

Yep it is, but with all the hardware the same we have hosts to use as spares or fall back on. They are cheap Dell R330's that cost around 1k each, so not really a big expense or hassle. All are the same model.

 

Just normal 2008 R2 Hyper-V hosts

2 SSDs per system, 1 x 600gb Intel and 1 x 2tb Samsung

Around 900GB user storage (We archive, resize photos, have a dedicated video server (planet E-Stream), Run FSRM to block certain files)

Backups are 4 x 2tb Samsung SSD on a dedicated server in another building yes (backup speed is key in our setup without RAID).

 

Hopefully you can see, we are not just using single disks and hoping for the best. We have designed the system around the fact that storage is easily migrated.

Edited November 13, 2017 by zag

[DJ-1701]

@DJ-1701 I live near Adastral Park, there are plenty of more talent folks around here that could provide a far better solution then I ever could.

 

Obviously that was just a jest ... though may I suggest that Adastral Park ('where world leading innovation happens') is a little ahead of the game when it comes to connections?

 

The only issue I would have with machines off site, is... I would not trust it on a single ADSL or a basic FTTC connection for speed or reliability. It was bad enough all the complaints I had to hear for being on ADSL alone... twice... and both times it wasn't in ITs hands.

[matt40k]

Obviously that was just a jest ... though may I suggest that Adastral Park ('where world leading innovation happens') is a little ahead of the game when it comes to connections?

It's worth a chat with some of the Ex-BT folks. Martlesham Heath the village they basically build for BT employees has "horrid" internet (I say horrid as folks have alot worse). A few of them live in the sticks which is why they setup that wireless internet solution.

 

Renting a server costs the same as buying it

 

Depends, if you're not upgrading your server every year, if your not factoring in power, human costs, you don't have any problems with it, you don't need to scale up \ down because of loads of students left\joined, you don't need to replicate your data within the same country but geographical separate, have ISO \ PCI or whatever compliance, the fact they give each user 1TB of space for free...

[mavhc]

Depends, if you're not upgrading your server every year, if your not factoring in power, human costs, you don't have any problems with it, you don't need to scale up \ down because of loads of students left\joined, you don't need to replicate your data within the same country but geographical separate, have ISO \ PCI or whatever compliance, the fact they give each user 1TB of space for free...

 

Who's upgrading servers every year? Power is part of the cost of renting.

 

Do many schools have that kind of rapid population change? Or ISO/PCI compliance?

 

Would be worth using the 1TB as archive for your previous year's work though.

 

If they offered an easy tiered storage server sync with 1TB per user, that would be nice, where your TBs of local server storage is a 1st tier cache (or 2nd if you have laptops).

[matt40k]

Do many schools have that kind of rapid population change? Or ISO/PCI compliance?

I might agree with you when I started my working, but I've had Middle Schools close, new schools open. Schools have physically disappeared and others have grown at such a rate.

 

As for "ISO/PCI compliance" - I won't wave the GDPR flag, just the fact you can't walk into a AWS \ Azure datacenter and walk out with the server without too much effort.

 

Replacing hardware frequency - you have to admit technology is evolving faster and faster. I wouldn't want to make 10 year plans beyond stuff will change and we will replace what we've got. Folks are already talking about quantum computing.

[MartinT]

My tuppence worth:

 

Host machines need disk speed and RAM more than CPU power, especially when hosting SQL or Exchange servers. All of my Dell hosts have RAID-10 disks with spinning SAS drives and caching PERC controllers. Server failover is still not as reliable as a hotspare disk, which allows the server to continue on smoothly until I can swap the drive out. Dell servers and PERC controllers are, in my experience, very reliable. I buy them used to stretch the capital budget. All hosts have 128GB RAM and run Server 2016 with Hyper-V.

 

I don't have a SAN as it's another single point of failure.

 

Single disk server? No thanks.

[geezersoft]

A well designed SAN is not a single point of failure.

[PotNoodleTech]

A well designed SAN is not a single point of failure.

 

A SAN with no single point of failure is a SAN beyond the price range of most schools.

[Gardinho]

A SAN with no single point of failure is a SAN beyond the price range of most schools.

 

Surely a SAN with a single point of failure is not actually a SAN!? I thought all SANs don't have a single point of failure...

 

We have been using SANs for years and I have to say that for us it has been worth the additional cost.

Edited November 14, 2017 by Gardinho

[3s-gtech]

If the SAN catches fire, then it won't work. In that way, it's a single point of failure

[JackCPickup]

Surely a SAN with a single point of failure is not actually a SAN!? I thought all SANs don't have a single point of failure...

 

We have been using SANs for years and I have to say that for us it has been worth the additional cost.

 

A SAN is a single point of failure...2 SANs isn't

[Gardinho]

A SAN is a single point of failure...2 SANs isn't

 

Not according to IBM.

 

"A SAN Volume Controller clustered system has several features that can be used to deploy a high-availability storage system with no single point of failure."

 

The SAN is specifically designed to avoid a single point of failure. Multiple disks, multiple network ports, multiple controllers, multiple power sources. All designed so that if one piece of hardware fails there is a layer of redundancy.

 

Minus the natural disaster damage of course!

[geezersoft]

A SAN consists of cabling / network / storage subsystems. A single subsystem is a single point of failure.

[Gardinho]

A SAN consists of cabling / network / storage subsystems. A single subsystem is a single point of failure.

 

Actually as I said all of the components in the SAN are duplicated to avoid that.

 

If the network or an external cable fails then those would be the point of failure, not the SAN.