Putting SIMS out in Azure

[pcstru]

Wrong, capita has no list.

Well, it's been a while since I worked much with Capita and SIMS but this looks very much like a list of both hardware and software requirements.

 

In reality they will likely give support up to a point but you might find that at some point they pin (say) performance issues on how your are hosting. If you are frequently running into issues they might then refuse you any support until your hosting meets their stated requirements.

 

From the other side of the fence that's not entirely unreasonable. If support and development time is being sucked up in quantity by a single customer, you will look at the situation very critically. It becomes unfair to other customers when you have to pour a lot of time into one who will not "play by the rules".

[matt40k]

By this rationale MSSQL on linux would be fine too, and SAMBA as the document server. The application wouldn't care.

 

Dms / docstorage is a windows service and mssql hasn't been released yet. You can use samba for the "s drive" ie the file server stuff

[localzuk]

Well, it's been a while since I worked much with Capita and SIMS but this looks very much like a list of both hardware and software requirements.

 

Doesn't look like such a list to me. Its a set of requirements, but not a list of supported platforms. No mention of "physical only", "Hyper-V but not VMWare" or "Azure not supported"... Just amount of RAM, number of cores etc...

[CyberNerd]

Dms / docstorage is a windows service and mssql hasn't been released yet. You can use samba for the "s drive" ie the file server stuff

 

ok docstorage aside. What about if you chose to use a clustered MSSQL environment for load balancing and failover? This is a much more likely situation for Azure/AWS/GCE based systems. I would imagine the capita engineer / 3rd party support wouldn't have too much experience of that and would happily pass the buck too.

[matt40k]

Again, Capita doesn't support sql, just the their database. If you have a cluster - and again, this should be hosted LA or a company not a school, you should have a experienced SQL dba. Having a good DBA is £50k, plus you need more then one (for failover) so you're talking about £200k bill in staff costs for true 100% uptime. What schools really want is say >10mins whilst the vm moves to over host

[CyberNerd]

Again, Capita doesn't support sql, just the their database. If you have a cluster - and again, this should be hosted LA or a company not a school, you should have a experienced SQL dba. Having a good DBA is £50k, plus you need more then one (for failover) so you're talking about £200k bill in staff costs for true 100% uptime. What schools really want is say >10mins whilst the vm moves to over host

 

 

I was thinking about the (absured) system requirements for such a tiny (10Gb) SIMS database. To minimize hosting costs you want to be paying for what is running. Given the SIMS is only heavily used at certain points of the day it would be most effective to run on a scalable system and only pay a minimal cost when it's under no load. I'm not too sure how Azure handles this and I was really thinking of some of the google services but presumably Azure can too.

[pcstru]

Doesn't look like such a list to me. Its a set of requirements, but not a list of supported platforms. No mention of "physical only", "Hyper-V but not VMWare" or "Azure not supported"... Just amount of RAM, number of cores etc...

There is mention of "computer and processor" and there are listed specific OS's which will run on that. There is no mention in the list of supported OS's of hypervisors or cloud hosting. Do you really think they should have to list everything that they do not recommend? That might be a rather big document.

 

You might argue with them that putting a hypervisor between the hardware they recommend and the OS they specify is neither here nor there, but if Capita find they have had to assign you a full time 3rd line/dev support and you are running some wierd set-up, they are unlikely to want to spend time eliminating that set-up as a potential factor.

 

Bear in mind this would only come into play if you are running into problems that require some expert investigation. If your azure hosting is working fine you won't even be talking to them. But when someone is assessing whether they want to run SIMS in production but using cutting edge hosting that Capita have not considered, they should think about the risk to their support and act to manage that risk accordingly. A chat with the lovely lads and lassies at Cardington would probably be wise before going down that road. Assuming that you will get unlimited support in such a situation and be able to argue that "well you don't say not to run it on X", would be ... unwise.

[matt40k]

If you follow this discussion to its nature logically technical conclusion you would end up with SIMS hosted. The reason it isn't is political, not technical. SIMS can be remarkably efficient if you tune it, if you build a warehouse you won't KO your indexes/caching when you run a full year attendances for the whole school for example.

 

The cost saves in terms of infra for a individual school, if they moved to Azure and auto-scaled isn't worth the effort. You'll get better cost savings with the economy of scale - ie 1 sql server shared with n number of schools

 

Your right @pcstru, there is a cost in terms of not locking into hardware, but you would still have to prove its not a application problem but your chosen hardware. You can't escape that. If you believe a certain device affects you software, then you've missed the whole point of the PC!!

[pcstru]

Your right @pcstru, there is a cost in terms of not locking into hardware, but you would still have to prove its not a application problem but your chosen hardware.

Unfortunately, in problem diagnosis the "proof" only usually comes at the end, when the problem is resolved. What happens is you call up, there is some investigation, they maybe ask for your data if they are feeling generous, internally they run up your data on their 'platform' and declare it works to spec, they then notice you have an odd set-up and say it must be your hardware/infrastructure and come back when you have checked that out and are running on kit that meets their spec. Contractually, they will be OK because they are not refusing you support for the product, they are just asking you to host in in a way that complies with their stated requirements before they will continue to throw resources at an investigation.

If you believe a certain device affects you software, then you've missed the whole point of the PC!!

Sure, I skipped the theory and went straight to the practical side where hardware that supposedly met specifications either didn't, or was driven by software that took liberties or was just badly written. These incompatibilities are why you suppliers like VMWare, Oracle etc will supply lists of hardware which they support (and have actually tested). When you get to nice expensive SAN's, you might even be restricted not only on what kind of drive you can use but which actual models down to batches of serial numbers. No doubt EMC and the likes just don't understand the point of standards like SATA 2.0, 'eh?

[robjduk]

Asked Capita for a quote to host my SIMS in Azure TWICE now to no response.

[leeshellard]

We have had a price from capita to host in azure it was £1500 for the first 30 users and £30 per year for each extra

[matt40k]

Wonder if I should log a call asking for a Capita to send its developers to this - An Introduction to Execution Plans and The Query Optimizer - maybe then you won't have to set compatibility to SQL2012 mode on SQL2014/2016 and won't need such overspec hardware.

[zag]

So nobody has it running on azure yet then......?

 

Surely the future of hosting database apps?

[localzuk]

So nobody has it running on azure yet then......?

 

Surely the future of hosting database apps?

 

Sticking our existing SIMS database in Azure doesn't appear to be the most sensible solution at present. The cost is significantly higher, as SIMS just simply isn't designed for cloud hosting really. Especially now that I've got 6 SIMS servers to host too.

[zag]

I guess we have to wait for them to write their web version then!

 

Its rather frustrating with Microsoft (and others) clearly pushing us towards the cloud and subscription services for Capita (and others) to catch up with the change in the industry.

[localzuk]

I guess we have to wait for them to write their web version then!

 

Indeed. Or, *sharp intake of breath*, we'll have to start looking at moving away from SIMS to one which has made the leap already...

[PotNoodleTech]

I guess you could host you're entire SIMS server VM in Azure rather than just the SQL instance?

 

It's not cheap though for the amount of CPU cores and RAM a SIMS server needs.

[Dave_O]

Reporting back from my testing to date:-

 

1. SQL in Azure is very expensive (too expensive) eg £3 per day and that's without much I/O. Abandoned this within a week.

 

2. Started afresh with £230 credit on Azure and started to do DPM backups, which to be fair (and after I'd got the caching drive sorted) works very well. However it does seem to be expensive (I suppose this depends on your perspective). See attached for details. As this rate I will use approximately £210 within the calendar month (£130 left with about £70 used) working out at around £2500 for the year. This (as you can see from the diagram) is based on around 5Tb of data on Azure most of which is data from the file servers.That doesn't include any restores only sticking stuff out there and storing it.

 

If the cost for the year was around £1000 then yes I would think it was a good solution. On reflection, rather than pushing everything out there I will probably be selective which will cut costs considerably.

 

My next test was just having blob storage and push Veeam backups out there, however my first venture into this a week or so ago when I backed up the DCs (which are quite small) was not brilliant in that it took 3 days to copy the vmdks up there. God know how long it would take the fileservers to complete. Yes you can use the seeding solution but not I think for Veeam backups.

 

All in all cloud storage is expensive, certainly too expensive for us at the moment.

[JackT]

Reporting back from my testing to date:-

 

1. SQL in Azure is very expensive (too expensive) eg £3 per day and that's without much I/O. Abandoned this within a week.

 

2. Started afresh with £230 credit on Azure and started to do DPM backups, which to be fair (and after I'd got the caching drive sorted) works very well. However it does seem to be expensive (I suppose this depends on your perspective). See attached for details. As this rate I will use approximately £210 within the calendar month (£130 left with about £70 used) working out at around £2500 for the year. This (as you can see from the diagram) is based on around 5Tb of data on Azure most of which is data from the file servers.That doesn't include any restores only sticking stuff out there and storing it.

 

If the cost for the year was around £1000 then yes I would think it was a good solution. On reflection, rather than pushing everything out there I will probably be selective which will cut costs considerably.

 

My next test was just having blob storage and push Veeam backups out there, however my first venture into this a week or so ago when I backed up the DCs (which are quite small) was not brilliant in that it took 3 days to copy the vmdks up there. God know how long it would take the fileservers to complete. Yes you can use the seeding solution but not I think for Veeam backups.

 

All in all cloud storage is expensive, certainly too expensive for us at the moment.[ATTACH=CONFIG]37435[/ATTACH]

 

Are you loading the credits straight into azure or are you going through a reseller first?

 

If you're loading it straight into Azure you need to remember that the pay as you go pricing are far higher than if you were to go through a reseller for the education pricing.

[Dave_O]

Are you loading the credits straight into azure or are you going through a reseller first?

 

If you're loading it straight into Azure you need to remember that the pay as you go pricing are far higher than if you were to go through a reseller for the education pricing.

 

It's a reseller with education pricing. Still doesn't change the fact that it burns the money away quickly

[PotNoodleTech]

Reporting back from my testing to date:-

 

Excellent work Dave. Very useful.

[zag]

Reporting back from my testing to date:-

 

1. SQL in Azure is very expensive (too expensive) eg £3 per day and that's without much I/O. Abandoned this within a week.

 

2. Started afresh with £230 credit on Azure and started to do DPM backups, which to be fair (and after I'd got the caching drive sorted) works very well. However it does seem to be expensive (I suppose this depends on your perspective). See attached for details. As this rate I will use approximately £210 within the calendar month (£130 left with about £70 used) working out at around £2500 for the year. This (as you can see from the diagram) is based on around 5Tb of data on Azure most of which is data from the file servers.That doesn't include any restores only sticking stuff out there and storing it.

 

If the cost for the year was around £1000 then yes I would think it was a good solution. On reflection, rather than pushing everything out there I will probably be selective which will cut costs considerably.

 

My next test was just having blob storage and push Veeam backups out there, however my first venture into this a week or so ago when I backed up the DCs (which are quite small) was not brilliant in that it took 3 days to copy the vmdks up there. God know how long it would take the fileservers to complete. Yes you can use the seeding solution but not I think for Veeam backups.

 

All in all cloud storage is expensive, certainly too expensive for us at the moment.[ATTACH=CONFIG]37435[/ATTACH]

 

Am I right in thinking your only backing up to azure (5tb!!) and not running a live sims server? I'd be very interested in feedback from the limited SQL test even if it only lasted a week.

 

Our sims server is currently 47gb big so i'm not sure how your report is relevant to this scenario?

 

Thanks for testing anyway, its great to see someone finally reporting back on its use.

Edited June 22, 2016 by zag

[sjatkn]

Thanks for this interesting thread.

 

My team are currently trying to persuade our SLT to allow us the time and money to get a proof of concept going for running SIMS in Azure. I know the PoC would not be too expensive as we can have it working only at the times when we are actually working on it. I would imagine that SLT would expect it to be running 24/7 if we decided to go ahead with it in production with the associated costs. What we would have is a reduction in the amount of hardware we had to support on premise especially if we moved more of our services in to Azure.

[Dave_O]

Am I right in thinking your only backing up to azure (5tb!!) and not running a live sims server? I'd be very interested in feedback from the limited SQL test even if it only lasted a week.

 

Our sims server is currently 47gb big so i'm not sure how your report is relevant to this scenario?

 

Thanks for testing anyway, its great to see someone finally reporting back on its use.

 

Not sure you have the right end of the stick or I have not explained clearly. The 5Tb represents the backup of the majority of servers, DCs, fileservers, app servers as well as SIMS (fyi the SIMS database is 12.2Gb) etc.

 

As for SIMS in Azure . You have to have a SQL instance, A server (basic machine 1 CPU 2 cores, 4Gb memory) on which this runs, an external IP and a VPN gateway. Once all of those were in place the initial cost for the first couple of days was £35. Left it running for a few more days, total cost after 1 week (7 days) was £76. If we exclude setup costs lets say £30 (minus 1 day). This means the cost was £46 for 6 days ie about £7.50 per day. This did not include any significant I/O. This equates to approximately £2737 for the year (who is going to switch SIMS off at night and over holidays?). Extrapolating and estimating bandwidth usage (we still need clear info from Capita about CPU and bandwidth usage in an average school to be sure) it will probably be around £3300 pa.

 

Given the SIMS application continues to pulls data down at random intervals, this will necessitate a change of usage pattern amongst users ie to reduce bandwidth they will have to log in and out of SIMS each time they want to use it rather than logging in at the beginning of the day and leaving it logged in. You can imagine how popular that will be.

 

At £3300 pa or even half of this, I do not consider this a viable option. Hence the reason for terminating the test.

 

There are numerous other technical issues that would need to be addressed before this would be considered a viable alternative. So for me this was an interesting exercise but will be shelved for a while.

 

I would really like someone else to corroborate my findings and hopefully show it can be done more effectively. For us SIMS will therefore stay on site until such time as Azure is more sensitive to education budgets.

[Dave_O]

Thanks for this interesting thread.

 

What we would have is a reduction in the amount of hardware we had to support on premise especially if we moved more of our services in to Azure.

 

Make no mistake you will still need significant on premise hardware for services that are nigh on impossible to put out there. In it's present guise SIMS (if that is what you are using for MIS) is, in my opinion, not a candidate. Also, depending on the Azure SQL model you use, you will still have to maintain the instance.