Solus 3 Change Requests Voting

[minimoo]

Hi Guys,

 

Capita announced on their website about voting for solus 3 CR's until the end of February. I've just got around to finishing going through all the CR's on solus3 and also logging 7 new change requests.

 

3 of which I care about a lot - and one which covers functionality I've been asking about since 2012 (when they BA said the product team thought it was a good idea and they'd put it on the backlog).

 

I don't know about anyone else but of all the things i manage - windows deployment, mac deployment, antivirus updates, windows updates etc - I'd say Solus is the one that I currently find most time consuming and most frustrating.

 

Of the 7 CR's I've added this evening, there's 3 that I think would save me (And probably some others a bunch of time and hassle), so would really appreciate some comments on here whether i'm unique or spot on - and votes either way accordingly

 

As the Capita website doesn't like formatting text with line breaks, I'll summarise here:

 

Provide ability to automate solus 3 via API/Powershell/SQL - https://myaccount.capita-cs.co.uk/ChangeRequests/ChangeRequestByReference/1702-3601854

 

I drafted this idea out on github @ https://gist.github.com/grangeway/270955419c26ccb729d5333c49b34bbe . We've pretty much automated everything around MDT/APPV - apart from SOLUS/SIMS. Our workflow uses MDT to deploy a SOLUS agent, which then never picks up sims as a target (for various reasons) until we manually go into the solus UI. Given Microsoft's love of powershell, and capita of dotnet - having some command line functionality that we could use to try and automate this process would really help in our environment - i.e. I'd write a script to remove an agent from solus once it's scrapped and deleted from AD for example.

 

Solus 3 – Cumulative Patch functionality - https://myaccount.capita-cs.co.uk/ChangeRequests/ChangeRequestByReference/1702-3601859

 

Solus 3 has downloaded 26 Database patches (since 1st January) - and 2 workstation updates (which are cumulative). That's basically one patch every 2 days if you deploy. Given microsoft's recent change to just doing cumulative for windows updates - it would be good if capita were to do similar - we don't get detailed information on whether a patch is important or not(seperate CR) so having a rollup would actually make life easier - as well as reducing the number of possible database versions capita need to support ( presumably at the moment that would be "1 + 26 factorial" if I remember maths correctly)

 

Solus 3 – Deployment speed - https://myaccount.capita-cs.co.uk/ChangeRequests/ChangeRequestByReference/1702-3601860

 

A "minor" tweak to deploy workstations that have downloaded the update first to try and reduce the time it takes to perform a sims update - which I've known a sims update to take several hours to "partially succeed" in the past

[matt40k]

The last one already happens... if you've configure it correctly.

[minimoo]

The last one already happens... if you've configure it correctly.

 

You can increase the concurrency however I don't think it deploys agents that have downloaded first for users that need a lower concurrency. I'm assuming that the reason that Capita set a low threshold for currency is that they don't feel the service can cope with high values. So either you have a low concurrency and optimise it - or have a high concurrency. I'll raise a support case with Capita to find out what the maximum concurrent value they recommend is - in our case, where we have 500 workstations, and server has GB link to network - a value of 500 would make sense if the service can cope and is optimised for multiple client connections.

[Max_Power]

Number one - Would be a nice feature, but wouldn't hold my breath for it being developed.

 

Number two - We don't deploy all workstation patches or database patches, just the ones that effect us so number two isn't really a problem. As all patches are included within the main Spring, Summer, Autumn and Winter releases anyway that follow on from the patch isn't that what your trying to achieve?

 

Number three - as mentioned above already possible.

 

 

 

Most important thing to us that we've asked to be developed is to be able to set priorities on stations. This will allow solus update machines with the highest priority first instead of what is generally a round robin attempt. Some staff work all year round, for their machines to update in the middle or at the end of a list of stations in excess of 400 is poor and unproductive. (if this is possible some other way i'm all ears).

[minimoo]

Most important thing to us that we've asked to be developed is to be able to set priorities on stations. This will allow solus update machines with the highest priority first instead of what is generally a round robin attempt. Some staff work all year round, for their machines to update in the middle or at the end of a list of stations in excess of 400 is poor and unproductive. (if this is possible some other way i'm all ears).

 

Isn't that a performance issue though? For instance, we have an Antivirus product with an admin console. If I hit deploy on the Antivirus admin console to send out an update immediately (as opposed to waiting up to waiting for it's hourly/daily/configurable check), it takes about 2 minutes to deploy to all 500 workstations. In contrast, I'm pretty sure that Solus takes over 30 minutes to achieve the same result. And I think this is where i'm coming from a different angle -> solus has a heartbeat check which runs over X minutes, you can force an update from a local computer. Therefore the (imo) maximum amount of time to install an update to a local workstation should be:

 

X minute delay for heartbeat + 2 minute "warning delay" + Installation time of update + Download time of update

 

The delay for the heartbeat should be able to be reduced by using the 'check for update' functionality on the workstation. The 2 minute delay can be reduced by hitting 'deploy now'. The download time can be reduced by waiting 24 hours before deploying the update so workstations can download and stage them ready. The installation time is kinda fixed - only difference there will be if pc's have HDD's or SSD's etc.

 

In that concept - I don't really see the point of being able to micro manage the priorities on a workstation. Equally if 400 workstations have already downloaded (staged) the update, it should be possible for the deployment to be pretty instant.

 

Re: number 3 - are you guys just talking about the concurrent settings? (i.e. settings -> solus3 -> deployments -> Agents -> Concurrent downloads ) and maybe a change to the deploymentserver config of buffsizepoolsize/messagesize of objects size) or something else?

[minimoo]

@matt40k @Max_Power - btw, i'm surprised neither of you mentioned the command line ui (re 1)?

[minimoo]

Number two - We don't deploy all workstation patches or database patches, just the ones that effect us so number two isn't really a problem. As all patches are included within the main Spring, Summer, Autumn and Winter releases anyway that follow on from the patch isn't that what your trying to achieve?

 

Regarding that comment - I also logged a CR about giving some more information to evaluate the importance. I think my point is - If Capita decide to push out a patch to 17,000 schools, they obviously deem it important enough to warrant the work to test it and publish it outside the normal 3 times a year cycle. Whilst some of the patches are obvious - others are not. And then there's the current issues with the "ctf export" patch that has been released 7-8 times now and capita are still trying to make work. At the point that capita decide that a patch warrants releasing outside of the normal release cycle to all school's, I'd make the assumption that Capita deem it an "important" patch. Solus emails that the patch is there - so applying them makes some sense. As sims re-release patches to change the DB version, fix bugs in a previous patch etc, I may skip ones I think i've applied before - My point here is that you can end up with lots of different database schema's/proc's running in this way - which is probably more likely to lead to issues

 

I think it also depends on your approach to updates - when Microsoft used to seperate out security updates, I used to wait 24-48 hours, deploy any IE patches - then look through the description of what the other patches were and deploy any ones that Microsoft deemed urgent/immediate - then wait a few days and apply the rest. That was to try and not end up with lots of patches over wireless at the start of a lesson. I often wondered whether that approach was more or less disruptive. Now microsoft do their cumulative patches - I wait 24-48 hours, hit google to check for people complaining about blue screens and then basically hit deploy.

 

End result is the same - but it's definitely a lot quicker managing the cumulative rollups for microsoft.... and i'd expect to find the same timesaving with capita. One cumulative patch that did 4-5 patches would save the space and time on the database server backing up an 18GB database multiple times to deploy each patch - as well as the time using solus. I currently have 122GB of sims backups from deploying patches today (and yes, I know there's a tickbox for that if you remember to tick it).

 

Across most of the requests the aim is really to reduce the amount of time/effort required to do tasks within solus.

 

Just had a thought - whilst I still think this is something that should be optimised within solus - rather then requiring lots of configuring priorities - looking at our last deployment:

 

514 workstations - 229 online, 276 offline, 7 failed, 222 successful

 

a) Capita deploy to 32bit and 64bit workstations seperately - and do 32bit *first*. This would probably be a problem even if implementing priorities for workstations - given that most workstations are moving towards 64bit... I'd be inclined to swap the order to do 64bit first... We have 0 32bit workstations on site - doing this first adds a 30 second delay to the deployment.

 

b) the 3rd workstation that solus told to deploy was:

 

2016-11-14 16:22:30.2488|lt002777|Info|Target version 7.166.38.0, expected target version 7.172.36.0, update will be deployed|ProcessAgentTargets()

2016-11-14 16:22:30.2508|lt002777|Info|Attempting wake on lan.|AgentIsAvailable()

2016-11-14 16:27:37.1945|lt002777|Info|Wake on lan failed.|AgentIsAvailable()

 

I'd probably be inclined to do an ORDER BY onlinestatus

 

c) Equally - to be fair to solus if I look at the logs for "update will be deployed" - the first timestamp is at 2016-11-14 16:22:29.9898 and the last is at 2016-11-14 16:26:06.6835

 

if it now only takes 4 minutes to send deployment commands to 514 workstations - @Max_Power - I've got to ask where you actually see a delay coming in - as it strikes me that the priority change you sound keen one would be a lot of work to save 2-3 minutes?

[matt40k]

@minimoo haven't installed SOLUS3 for over 4 years

[Max_Power]

@minimoo I think we are both looking at this from different angles. Every school is different and this is possibly why not all things get implemented via this voting system Capita have in place. What one school deems important, others may not and even then Capita may deem it not possible.

 

All I want is a quicker way to update SIMS via Solus, I appreciate that is has come on leaps and bounds since the days of SOLUS 2 etc but there is still room for improvement, whether that be performance related or function / feature related.

 

While it may only take 4 minutes to send out the update command to your 500 odd stations, that's not the length of time for the update to complete? Once the command is received the stations need to update, do all your stations update within that 4 minutes or is this the 30minutes you mention further up the threa? I'd have to go through the logs from the last rollout but it can take a good hour or so for all 400 stations to update and be in a working state. Maybe I have some configurations wrong somewhere if all your 500 are ready to go and work within a matter of minutes?

[minimoo]

@Max_Power Equally i'll need to watch next time - I'm used to expecting sims updates to take for ever i.e. it used to be the case if I kicked solus3 off at 3.45pm (the second the bell went), that the deployment would still be in progress at 18:00 - or just finishing as I leave. So my times were from looking at deployment log of last update. In solus 3 - Can you check settings -> solus3 -> deployments and under agents see if you have 'auto-download' ticked?

 

I spend a fair amount of time doing development, so i'm probably also coming at this from a technical angle. I also feel that solus3 takes up more time then it should do and I personally think it's a shame you can't comment on CR's - as sometimes when reading them it's possible to spot something related.

 

I suspect I'm probably uniquehere - but if I go back to April 2014 - 25% of the support cases we have raised are on Solus 3. Since April, if I look at calls we've logged - 30% were for solus3, 18% were for existing patch requests. 16% of calls were for bugs/issues that required a patch/application fix (we've got an FMS issue we hit most years at year end and have to get a modified version of the same patch for each year). 25% were on some novaT/NovaP issues and trying to create some reproducible test cases - and the remaining 10% of the time were enquiry/queries. Whilst I'd hope that's a unique picture for us - it does mean that 50% of the times I contact the helpdesk are either regarding solus 3 or requesting a patch that already exists, isn't on solus3 and I need to ask for manually. Equally, if I'm not unique here - you could drastically reduce the load on the helpdesk by looking at the patch management process.

 

In terms of solus, the only things we did/do are:

 

a) we push out agent via vbscript from capita (as back in the initial days at least, pushing out an agent via AD caused WMI to crash sometimes, taking down the solus service, and required a laptop to be on mains power for the scheduled task the solus service created to run to install the agent (or actually i think install dotnet when that was required)

 

b) we changed the default port from 55432 to 48000. This was due to windows sometimes using the port for existing TCP connections causing the solus service not to be able to listen on the port ( https://support.microsoft.com/en-gb/help/929851/the-default-dynamic-port-range-for-tcp-ip-has-changed-in-windows-vista-and-in-windows-server-2008 ). We originally installed solus3 in 2011 - Microsoft added a hotfix in 2012 to allow people to exclude ports from the default range ( https://support.microsoft.com/en-us/help/2665809/you-cannot-exclude-ports-by-using-the-reservedports-registry-key-in-windows-server-2008-or-in-windows-server-2008-r2 ). I'm not sure if solus' installer runs that netsh command, but I could probably move to using the default port if I ran that command that didn't exist back in 2011

 

c) In October/November 2016, Capita made a change to the solus deployment server to increase a buffer/max message size - but that was due to a change in the latest 3.56.1 release I believe. I suspect they'll change the value by default in the next release

 

d) we changed the heartbeat timeout from 5 minutes to 13 minutes [This was orginally done in 2012 to avoid tolerance issues when performing FMS updates as we had 3 FMS databases running]

 

e) we changed concurrent downloads from the default of 25 to 99

 

f) I set up solus to auto-download updates to workstations - but never auto-deploy - I want files to ready as soon as possible for when I deploy, but don't want any surprises.

 

I normally wait 24-48 hours after the release to give workstations a chance to download then hit deploy.

 

What I think used to happen is solus would kick off 25 updates at a time, then if someone turned off their PC in the middle, literally wait for 30 minutes (until the deployment timeout interval was hit) before deploying the next batch - so it would occassionally "get stuck". I think capita may have fixed *that* - and if so, that's where I think the priority thing you wanted no longer makes sense. The *only* time it would make sense to have priority would be if you were going to deploy the update within seconds of the release (i.e. you are not allowing a window to download to the agent).

[misternormal]

I would just like it if solus3 worked, since its introduction we end up manually installing SIMS on about 50% of PCs. Even if it appears to be working and recognising a PC it can still take more than an hour to update, which if you are waiting to take a register is useless.

[Esteban_Child_of_the_Sun]

I would just like it if solus3 worked, since its introduction we end up manually installing SIMS on about 50% of PCs. Even if it appears to be working and recognising a PC it can still take more than an hour to update, which if you are waiting to take a register is useless.

 

In the latest version of SOLUS 3 (3.12.41) you can specify a priority for workstations so computers that are needed to take registers are done first.

 

Also, Capita confirmed at a regional user group that 3.12.41 was the last currently planned release of SOLUS3 so bar critical bug fixes that aren't caused by local environmental issues there will be no more updates ever to SOLUS3

Edited February 22, 2018 by Esteban_Child_of_the_Sun

[Rawns]

Also, Capita confirmed at a regional user group that 3.12.41 was the last currently planned release of SOLUS3 so bar critical bug fixes that aren't caused by local environmental issues there will be no more updates ever to SOLUS3

 

Interesting...

 

At BETT I was assured by Capita reps that as SIMS 7 has no end of life yet, SOLUS 3 would also be "business as usual" and will actively be developed as it also has no end of life date.

Edited February 22, 2018 by Rawns

[David44]

Why don't people just run SIMSLoad.exe to update SIMS? I have never understood the point in SOLUS. It seems that it is trying to be a package deployment product that only works for deploying SIMS/Capita products (and from what I have read, it isn't very good at that). Schools will already have a deployment system, what is wrong with deploying SIMS with that and then when there are updates to SIMS just run SIMSLoad.exe on the workstation to download and install the new packages? Takes no more than 5 minutes

 

We actually run SIMSLoad.exe as part of the login script, so when we build a new workstation an old SIMS package gets installed and then when someone logs in for the first time (which will usually be a technician rather than an end user) SIMS gets updated to the latest version.

[matt40k]

SIMSLOAD is horrid and needs to have die when we all moved to Windows 2000.

 

- Users should not be able to write to Windows directory or the program files directory.

- IT should be proactive not reactive and push software, you shouldn't have to wait 5 mins for an application to update just to spend 2 mins taking a register

 

Surprised Capita hasn't spoke about using the Windows Store

[David44]

No, users shouldn't have to wait 5 minutes for software to update, neither should software crash with meaningless error messages, freeze for all users because one user is running a certain report, launch separate processes with completely different GUIs that look like they haven't been updated since the early 90s etc. but this is SIMS we are talking about. Five minutes for an update is the best we are going to get. People further up the thread are talking about upgrades taking hours and even then not working and this is not the only thread with these stories either.

 

I take your point about writing to program files and the Windows directory (although I don't think the SIMS installers write to the Windows directory on an upgrade anyway, do they?)

[matt40k]

Take a look at C:\Windows\sims.ini. SIMS moves at a glacial pace, look how long it's taken to get folks to adopt the free stuff that Capita are activity pushing? Microsoft suffer from the same thing, Win10 is what, 3 year this year, still not widely adopted. This is before anyone comments on how long it takes Capita to develop anything - bulk import UDF anyone? I release an open source app, what 5 years ago, physically had the code on github for anyone to grab (using the SIMS API \ business objects too) - still slated to be release later this year.

 

FYI there are other MIS system out there

 

PS: I have no idea what you mean, I mean surely everyone knows what those lovely long codes mean - especially when they link to awesome up-to-date KB articles... oh wait..

[minimoo]

In the latest version of SOLUS 3 (3.12.41) you can specify a priority for workstations so computers that are needed to take registers are done first.

 

@Esteban_Child_of_the_Sun the priority feature is pointless (imo) for a single school. We deploy solus to 500 PC's (and having pre-staged) the update, it takes about 4 minutes for the log file to cycle through pushing the update out to the 500 pc's. Given that, you would spend longer trying to prioritize the PC's then you would save by the end of it!

 

If the solus 3 console states that the update takes about an hour for us; 20-25 minutes is doing the DB updates, document server etc - a fair chunk of that time made up by '2 minute waits'. 5 minutes is spent deploying workstations, and 30 minutes is spent waiting for a timeout of failed workstations.

 

In terms of 3.12.41 being the last major update, I'm definitely not surprised there. In terms of features that is. I do think capita needed (and still need) to spend some time on optimizing the process and fixing the flaws that stop people using solus effectively for a deployment.

[Cache]

I would disagree about the priority feature being worthless.

 

I have it configured so that my own, the server and a couple of admin workstations are updated first - that way if it is just a single workstation update to roll out I can be on importing the report files and checking things have worked correctly and certain people can start work again while the update continues to roll out. As I do this after school at 4pm it matters less about classroom workstations being updated as a priority and I'm happy for them to get done as Solus 3 decides.

[minimoo]

@Cache - I think my point was more, IF it takes 5 minutes to trigger the update on 500 workstations, whether I'm at the front or the end of the queue doesn't matter; It can take a minute or two for the agent to detect the install has finished. I always make one of my first tasks after logging in to check that a teacher can still log in - there was one update once where capita either broke the 'take register' box on home page or changed the permissions; I could use sims fine, the 90 teachers couldn't in the morning - sims crashed on them. Since then, I always find a member of staff to make sure everything is happy.

 

I guess it all depends on what is important to different people - but I waste a lot longer in a day on sims then I do with the actual bit you can prioritize on a workstation deployment.