Delays to SIMS releases
SIMS 2009 Autumn Update Release - Change of Date
An issue has been discovered in our final release candidate for the SIMS 2009 Autumn Update release. Due to the impact of resolving this over the Christmas period, the release date is now planned for COB on Friday the 8th of January to allow time for a complete regression test of the software.
We apologise for any inconvenience that this delay may cause.
Please feel free to email me if you have any concerns about this on firstname.lastname@example.org
SIMS Product Manager (Technical and Releases)
SO it appears that the move to seasonal release names has not really seen any improvement as the Autumn 2009 Update will now not be release until 2010!!! and COB on friday 8th in Reality means we will not be able to do anything on this until the 11th January
Maybe some of you guys would like to ask Capita if you could take part in their UAT Testing programme, using your datasets?
With over 21,000 schools using SIMS it is very difficult to ensure that testing finds all of the issues in a new release, which might affect just some schools. In the past, I have found problems at UAT Testing which have not been evident in the Testing carried out prior to UAT (and there are several layers of this). Nor have they been evident to other UAT Testers. i.e. the issues discovered do not affect all establishments and, to all intents and purposes, the Update seems to be fine. It is often only when a User such as myself (and other volunteers) actually UATs the release on our own data, that these anomalies can arise.
On one recent occasion, our dataset would not even upgrade in readiness for UAT. This issue had not been evident on any other datasets that had been tested. As a result of finding this issue at UAT, a fix was produced when meant that the Upgrade was not delayed at all because of this and, of even more relevance, no school experienced the problem after release of the Upgrade!
I, for one, would prefer the delay of a Release if a bug has been found by a fellow tester that affects some, but not all, instances of SIMS. After all, we might be one of the schools affected next time!
What is involved in the UAT testing? Do you do it on your server or do you send Capita your database and they test it?
I agree with the approch taken by Capita to delay a release when a problem is discovered. This main issue I have is with the number of times that this is happening!!! I can't remember the last time that Capita actually released a product on time.
Originally Posted by Sivadam
The worry I have is with the clear difference in the testing done prior to the UAT. Are their own internal processes actually that basic? I often undertake Beta testing on products so am not simply complaining for the sake of it
You can get the release made available for you to download on to a test machine. Or you can send the dataset down to Capita and attend one of their UAT sessions in Bedford. I do the latter.
Your dataset is loaded on to test machines and then upgraded to the new release. You are presented with scenarios of the new procedures in SIMS and test to see if they work as they should. It is also important that you test the other parts of SIMS that the new functionallity could affect.
A Feedback Spreadsheet is provided in which you log any issues that you have found.
The results of the testing are analysed to see if the release is fit for purpose.
At the last testing I spotted that the Send message, from a pupil to 'Teachers of Pupil', was only sending to the Pupil's Teachers for that day. This did not prevent release of the Upgrade but did go on record as something to be looked at for next time. I have referred to this in another thread, which is on SupportNet.
I have found this UAT procedure invaluable in getting ready for a new Upgrade. I have had an insight into new functionality and can inform our staff in advance of such new procedures, so that they are ready to utilise any innovations as soon as they are available.
An obvious & recent example was the NON-EDI Results in GCSE Exams (August release) - absolutely brilliant! We had our exact Headlines figures seconds after the Results were uploaded, because all extra results were already on the system!
I am sure that the Capita staff share your misgivings. I believe that they do have different datasets that they test on prior to giving it to us guinea pigs at UAT.
Originally Posted by Mcshammer_dj
All I can say is that things have improved greatly since the move from db to sql happened - when I was in the forefront of those complaining.
But the problem is that different schools want different things from SIMS and we all use it in different ways. If several 'real' schools' datasets do not reveal problems, and then suddenly one does, how do you mitigate for such a scenario?
So do you get any form of benefit for this work that you do? I would guess that if the licence cost was lower for being in the UAT process then it may have certain benefits. Given that I have 70 databases to support I really do not have a great deal of time to spend looking at the UAT process.
We instead expect that the cost of the SLA allows for capita to deliver the product that we require, and also expect that the timescales for that delivery be met. Lets be honest some products (SM7) have consistantly been removed from the process meaning that some of the ways that the product works are still akin the the common platform days.
Equally it is worth noting that even once a product is finally released then we all test it again and often find problems are still present leading to the patchwork quilt that follows a release
I'm afraid that you are expecting more than we can achieve and when faults are found we have to delay. Perhaps it is possible to do better and the team will try to do so.
We work 29 different returns into our releases that are defined at different times of the year and sometimes require significant clarification. We have to weave that into fixes and enhancements and deal with operating system changes. The delay is actually around 10 working days.
The returns for January were issued in October so as far as I am aware there isn't a pressing need to roll this update out to all schools.
hey Phil I know that you all work hard to continue to deliver a service and we generally do appriciate all you do...
Originally Posted by PhilNeal
It just seems so disheartening that the move to seasonal releases to try to make it easier for us to deliver releases to schools without seeming to be months late has mean that we are again in the position that we tell schools that we are delivering the autumn 2009 update in 2010!!
Not always the case as often when we find a problem we will be advised to apply the update as that may fix the problem!!
Originally Posted by PhilNeal
@Sivadam RE UAT testing:
Do Capita pay your school for doing this? It's just that spending time plus travel doing something that should really be included in the support/maintenance charges, our HT would expect to be reimbursed for my time and expenses.
No they do not.
Originally Posted by teejay
The only benefit that we/I gain, is an increased knowledge of the product and advance sight of what is coming, so that the school is in a better position than most to take up and use the new functionality. The school is well aware that we are highly regarded in our use of the SIMS Software mainly as a result of the breadth and depth of my knowledge. We do not have the issues that some schools may have of nobody having an thorough working knowledge of all the funtionality in SIMS. In effect, I use UAT in advance of an Upgrade in a similar way that others would use Training Courses for the new functionality after the release of an Upgrade.
I would also like to think that my opinion, on the way some of the new SIMS functionality develops, does hold some sway - even if this is only to a small degree.
@Phil - If it's legal, would it help if schools gave you permission to retain their data when they send it in, so that you can use this for testing.
I know the changes we make over time lead to more issues so static data isn't as great as a full UAT test, but having a more diverse source of test data might be more beneficial.
I often find odd and obscure problems on our systems and inevitably this leads to the need to send my data in, which takes a long time from here. I then work alongside the support team, both working on virtual setups to fix the problems. I'm sure this must happen to others as well, and though we may not be able to spare the time to test the new upgrades i'm sure there must be schools out their that wouldn't mind you retaining there data so long as the necessary security precautions were taken.