What No Audit trail ?! - Safeguarding

[Geoff]

@localzuk I'm surprised that's not a legal requirement given the implications to be honest. As I've said I don't use Sims and haven't for years but could that not be mitigated by user ACLs? I assume there's a system within SIMS whereby you can lock down access to aspects of the application and the pupil data within on a per user or role basis?

[localzuk]

@localzuk I'm surprised that's not a legal requirement given the implications to be honest. As I've said I don't use Sims and haven't for years but could that not be mitigated by user ACLs? I assume there's a system within SIMS whereby you can lock down access to aspects of the application and the pupil data within on a per user or role basis?

 

There are ACLs there, but they aren't really that much help in a child protection case. If a teacher were looking at a student's address with no need, for example, they would be able to as part of them being a teacher and them sometimes needing that access for legitimate reasons. Audit logs would provide a trail to look at to see that teacher looking at things.

[CAM]

Some roles grant permissions to view particular types of data in other areas of SIMS. For example, my role as Cover Manager and Statutory Returns Officer grants me enhanced access to STAR and Personnel (but I still cannot view contract and pay details because they are restricted).

[pcstru]

@pcstru, you want some reasons why auditing should be in place?

How about being able to determine who has been accessing details in the case of a child protection issue? Eg. A child has been targeted somehow, and there's a hint that it may be a member of staff - being able to look at who accessed that child's records may be of great assistance in any investigation.

This falls into the categories I posted here. If they should have access, i.e. they are legitimately employed to do a job and need to be granted access to the data via permissions, then what will a log tell you about that access? How can you tell if their access was legitimate or not? If the 'hint' (whatever that might be) is sufficient to cause suspicion which is a safeguarding issue (i.e. you believe the member of staff represents a threat to the child) then access should surely be removed entirely as should the member of staff? You don't wait to see if the risk turns into an issue just so you have some evidence.

 

How about to audit what us IT admins are doing in SIMS? We have unfettered access to the system. Sure, we could poke around in the SQL database itself, but most don't do such things. Having a proper audit would help some way to showing that IT staff don't misuse their privileges.

The fact that you acknowledge that there is a way to drive a coach and horses through what would be a massive piece of development tells me it is utterly useless. You trust people not to access the database directly yet you think they are stupid enough to know they can have that access but yet they do it through the software that they know is monitoring them when they are (knowingly!) up to no good? If a company suggested that this was effective as a security measure they would be rightly laughed out of the forum (witness the criticism (rightly) of Impero). If you don't trust your technical staff then you are going to need to act in a way that entirely bypasses them if you are into gathering evidence. If you are gathering evidence, there are other, probably better ways to do it but ...

 

There is a disconnect also between gathering evidence when you have a reasonable suspicion that staff are a safeguarding issue - i.e. you suspect they are an actual risk to children's safety. If the suspicion is reasonable, you remove them, you do not leave children vulnerable to them while you gather more evidence (more because you must already have some). That would effectively be using the students as bait.

 

Let's also look at some other aspects. How does the school deal with access to printed reports or someone looking over a shoulder at someone else's screen? If you build in, at huge expense, read logging to the application & database, how do you cover those cases? I appreciate that not being able to solve all the issues might not be a reason not to solve any of them, but when you look at the actual problems here, there are other solutions that are equally or more effective given the possible circumstances that can arise. Putting trust in a half baked technical solution would just be ... half baked.

[localzuk]

This falls into the categories I posted here. If they should have access, i.e. they are legitimately employed to do a job and need to be granted access to the data via permissions, then what will a log tell you about that access? How can you tell if their access was legitimate or not? If the 'hint' (whatever that might be) is sufficient to cause suspicion which is a safeguarding issue (i.e. you believe the member of staff represents a threat to the child) then access should surely be removed entirely as should the member of staff? You don't wait to see if the risk turns into an issue just so you have some evidence.

 

Gathering evidence after the fact can aid in disciplinary procedures, and aid in criminal cases. Not having it can limit those things.

 

Let's also look at some other aspects. How does the school deal with access to printed reports or someone looking over a shoulder at someone else's screen?

 

Printed reports are sent via print and release systems in every school I've worked in - ie. they are kept confidential. Here, printed reports are seen by the teachers writing them, the 2 admin members of staff who collate and quality check them, and then they are sealed into an envelope and sent home.

 

Those people who do confidential stuff on computers are in a different office, so that those who can "look over their shoulders" are only those with the access to that information anyway.

 

If you build in, at huge expense, read logging to the application & database, how do you cover those cases? I appreciate that not being able to solve all the issues might not be a reason not to solve any of them, but when you look at the actual problems here, there are other solutions that are equally or more effective given the possible circumstances that can arise. Putting trust in a half baked technical solution would just be ... half baked.

 

Arguing against one solution because there are other issues is a case of "whataboutery", and irrelevant.

[pcstru]

@pcstru

I don't think you need educating but again the point of this thread isn't to make a business case or to justify why not to make one. It's to highlight something missing from a product from the point of protecting the students. Step back and think about where we are trying to do here. I also think there are enough replies to this thread to see that people have requested this. I do agree with you though some transparency on decision making from Capita might be beneficial.

I don't see how you can run a software development business like a schools MIS system without some some kind of rational "business case" driving the development of features. That would seem to be an argument that says "make software that no one wants". There is a business case for having user CRs but each CR then needs to be considered in terms of it's cost and the benefit. If there is a huge cost but no real benefit then why would customers pay for it?

 

If this is a genuine "statutory" need then it should be a feature of all MIS suppliers. Read logging certainly isn't. Even auditing changes in a way that is application aware isn't covered well by the big players and I doubt it is a factor in many purchasing or renewal decisions (it wasn't in ours which was just ~2 years back).

 

Still, if people here are right - all MIS suppliers will need to do this or we will not be able to buy their products. If it is a serious safeguarding issue then that is surely what it will come down to in the end?

[pcstru]

Gathering evidence after the fact can aid in disciplinary procedures, and aid in criminal cases. Not having it can limit those things.

Evidence in a criminal case would need to be beyond reasonable doubt. I struggle to see how a log could be that evidence when someone could lean over and view data on a screen or printout. If you are talking about mere disciplinary and you have the luxury of gathering such evidence, then there are other, probably better ways to do it but it is not safeguarding if you are gathering evidence to prove suspicions you already have.

Printed reports are sent via print and release systems in every school I've worked in - ie. they are kept confidential.

Oh come on. Print release simply means it is not left lying around on the printer - usually (print release doesn't stop someone starting a print and wandering off) nor does it control what happens to the paper after printing.

Here, printed reports are seen by the teachers writing them, the 2 admin members of staff who collate and quality check them, and then they are sealed into an envelope and sent home.

 

Those people who do confidential stuff on computers are in a different office, so that those who can "look over their shoulders" are only those with the access to that information anyway.

It is interesting that here you are OK with policy, procedure and trust keeping the information secure, but in the context of an application, you think that those are deficient.

Arguing against one solution because there are other issues is a case of "whataboutery", and irrelevant.

I'm not. I'm arguing that the enormous cost is not justified because the problem can be and already is better addressed by other means. I'm probably about done with it but I'll happily take a bet with anyone that no company providing schools MIS systems will commit to offer "read logging" (a method of recording access to data by application users) in the next 2 years. Anyone who believes that this is a statutory requirement mandated by safeguarding should consider that a safe bet and take my money. I don't expect any takers.

[elsiegee40]

@pcstru you will argue against until the cows come home. You will also fail to convince others that there's no point.

 

If others wish to continue the argument with you, somewhat pointlessly in my opinion, that is up to them.

 

From my point of view, any evidence that helps to build a case when investigating an issue ... or to disprove a case... is useful. Not having that evidence could be crucial in protecting a young person... or in protecting the career of an innocent member of staff. Not all evidence has to be beyond all reasonable doubt... sometimes 'on the balance of probabilities' is what is needed ... the law enforcement agencies then use it to help them build the criminal case which is rarely hinged on a single piece of evidence.

 

And with that, I am happy to continue the conversation but not to continue circular arguments. I don't doubt you will have the last word simply because you seem to need to

Edited February 8, 2016 by elsiegee40

[CyberNerd]

Interestingly; Capita claim that their hosted service does allow auditing:

 

Q 6.4 – Are audit trails in place

enabling users to monitor who is

accessing their data?

 

YES Audit trails are available to administrators.

 

http://campaigns.capita-sims.co.uk/sites/campaigns.capita-sims.co.uk/files/cloud_services_and_the_data_protection_act.pdf

 

How is the hosted service different in this respect ?

[pcstru]

How is the hosted service different in this respect ?

It was built in from the beginning? The phrase "their data" doesn't quite describe what you think it does (i.e. you can see who is accessing data but not with any granularity)? It could be any number of things.

[CyberNerd]

It was built in from the beginning? The phrase "their data" doesn't quite describe what you think it does (i.e. you can see who is accessing data but not with any granularity)? It could be any number of things.

 

Not according to Capita. They told me at BETT it's just SIMS running over a VPN over RDP.

Perhaps their idea of auditing is "yes someone access something at this time" we don't know what they accessed or who accessed it. Anyway, they managed to self-certify their cloud services this way so presumably stand alone SIMS meets the same requirements.

[pcstru]

@pcstru you will argue against until the cows come home. You will also fail to convince others that there's no point.

 

If others wish to continue the argument with you, somewhat pointlessly in my opinion, that is up to them.

 

From my point of view, any evidence that helps to build a case when investigating an issue ... or to disprove a case... is useful. Not having that evidence could be crucial in protecting a young person... or in protecting the career of an innocent member of staff. Not all evidence has to be beyond all reasonable doubt... sometimes 'on the balance of probabilities' is what is needed ... the law enforcement agencies then use it to help them build the criminal case which is rarely hinged on a single piece of evidence.

 

And with that, I am happy to continue the conversation but not to continue circular arguments. I don't doubt you will have the last word simply because you seem to need to

 

I think you are just repeating claims that I have tried to address by considering the actual detail of them, but this time you feel the need to mix in a little ad hominem. So you are right the 'argument' has become circular and other than a response to anyone that wants to take up the bet, I guess it's back to lurk mode for me.

[pcstru]

Not according to Capita. They told me at BETT it's just SIMS running over a VPN over RDP.

Perhaps their idea of auditing is "yes someone access something at this time" we don't know what they accessed or who accessed it. Anyway, they managed to self-certify their cloud services this way so presumably stand alone SIMS meets the same requirements.

If it does then they should find it easy to provide the functionality for local installs (probably at some fairly significant cost) or perhaps if schools think it is actually a requirement, then they need to move to a hosted service.

[Jonah]

I think it would be interesting (and would probably therefore tie in to the "people aren't requesting this feature" excuse) to see just how many senior leaders in school simply *assume* that this functionality is in place within SIMS, and until they ask the question of IT (or whomever) will be none the wiser?

 

As someone mentioned previously, if it was mentioned to them that there was no way of checking who might have changed what record, I reckon there'd soon be quite a large amount of schools piping up about it in concern. Just because they haven't as of yet could simply (and, most likely) just mean they haven't yet needed to ask the question.

[GREED]

Not according to Capita. They told me at BETT it's just SIMS running over a VPN over RDP.

Perhaps their idea of auditing is "yes someone access something at this time" we don't know what they accessed or who accessed it. Anyway, they managed to self-certify their cloud services this way so presumably stand alone SIMS meets the same requirements.

 

Perhaps put this directly to Capita or @PhilNeal will be the quickest way to get an answer.

 

Having also completed this self certification form for the DfE, I suspect there is actually context to the question, which would explain the differences you bring up.

[vikpaw]

Sorry, catching up on a 4 page thread.

Some thoughts:

 

The option is there in current SIMS but it's turned off by default.

Tools > Data Change Management I think.

That's only changes. It doesn't log who accessed what.

 

Yep, that's my understanding. Data Change Management I think was more of a tool for third party integrators to basically flag a delta when info had changed.

 

Has that LOG in system manager 6 ever worked?

I have used it in the past but to stop it from crashing you had to be very specific and drill down to search for a particular module that was accessed. I got some results in a test, but never managed to use it in a real scenario.

 

I recall a time where change requests were just deleted if the Capita support teams didn't like them. Does this still happen?

Probably. Also there was some rule applied when SupportNet became My Account, so a lot of the old ones were harvested!

 

Out of interest, what auditing processes do the other MIS systems have? Step forwards please MIS reps.

Double First Engage actually has a pretty decent feature for this. I'll expand more further down.

 

I must be misunderstanding that too then. SUSER_NAME() will return the user in the current security context, which should be the DB Login (for SIMS). That seems to be how it works here so I'd be delighted to hear what I'm doing wrong.

I can't remember exactly, but I used to have a couple of scripts that would tell me which user was connected to the database and from which machine. I'm not sure if this could be combined with access to specific modules / tables. Probably not as you say because the 'work' is done through the app and a central db user.

 

There are ACLs there, but they aren't really that much help in a child protection case. If a teacher were looking at a student's address with no need, for example, they would be able to as part of them being a teacher and them sometimes needing that access for legitimate reasons. Audit logs would provide a trail to look at to see that teacher looking at things.

Exactly.

 

My most common scenario was who changed / deleted a behaviour record? Staff have legitimate access to edit this data but if it's done in malice or to cover something up then we want to track it. We do have suspicions, but monitoring them, or tracking them, videoing etc. aren't going to do the trick. We're talking about random one-off incidents, that quite possibly are not detected until quite some time later.

 

If Capita were to implement auditing in any module that is one where it has been raised a few times that users would like to see it. We provided feedback at all stages even for the big Behaviour Management module review, but I don't think it made it in.

 

Another area where monitoring a change is useful is the attendance figures. SIMS is supposed to record this, I thought it was a legal requirement, but I never got it to work properly. For starters the reason for change was never properly implemented, it always defaulted to 'Entered in Error' and the user couldn't change it. Also, I'm sure through certain routes, the system would log the change as the original class teacher of the class, not the actual user who did it. I can't comment in full as it's been a while since I tried to monitor this area, but it was always a sore point for me. It could be a simple as someone getting a detention for too many lates, or as important as a student not being able to graduate because their % attendance was not enough for the year. Retrospective tracking would help.

 

@CAM et al , if you're interested in how it can be implemented :-

 

We use Double First Engage and it has automatically active Attendance Change tracking and a report that will show the exact change and who did it. On changing an already saved mark, the user is prompted to supply a reason. I think this is a feature you can turn on, we did naturally!

 

Engage also has a pretty nice Auditing area where you can choose which area you want to monitor and turn on auditing on a per table / module basis.

When you view the report you can choose a date range and also drill down to a specific user if required. I don't have a lot turned on at this school, but for today's AM Session Attendance I've got ~2100 records.

 

You can see inserts and modify, in the attendance reports the changed attendance report shows both the original mark and the change together.

Here there are changes to old data being done as they are tracking missing marks and updating them from paper copies and notes prior to extracting data for the term reports.

 

 

Hope that's of interest to some. Sorry Gary - rubbing it in

Edited February 9, 2016 by vikpaw

[CAM]

@vikpaw: Doesn't SIMS have the Reason for Attendance Mark Change routine?

[PhilNeal]

@GREED thanks for your prompt.

 

With regard to our hosted SIMS offering - we can tell when an individual logs on to the service and therefore who has access to the data at specific times. What we cannot do is tell what the individual data is that they have looked at.

[vikpaw]

@vikpaw: Doesn't SIMS have the Reason for Attendance Mark Change routine?

 

I don't know. Does it... ? Not actively used it for a long time.

[CAM]

I don't know. Does it... ? Not actively used it for a long time.

 

I tried opening it to see what it audits but it crashed....

[mrwoberts]

Vote on this Capita change request if you would like add your support

 

https://myaccount.capita-cs.co.uk/changerequests/ChangeRequest/2f03420b-95d4-e411-8edd-005056150016

[ijk]

A log recording the actions of any user of any system does not prove that the individual assigned the credentials used to access the system carried out those actions.

[SwedishChef]

No but any aup of worth would state the user is responsible for their login details

[vikpaw]

Plus a list of users of a given set of credentials is way smaller than - "...well, it could be any member of teaching staff, plus most of admin, oh and any TAs and student teachers that we 'trust'...."

[SwedishChef]

Love it @vikpaw