Data Transfer (Was Changing from SIMS)

[pcstru]

(moved from http://www.edugeek.net/forums/mis-systems/136204-changing-sims-8.html#post1168990) : hopefully the discussion can complete here.

 

My focus too was the fact that schools do not want to share the data they have because they are often unaware of the

quality of what has been written, and I refer you to the point about behaviour information. How also is it misleading if I am referring to my own experiences given this is an opinion, as well as a fact in the schools I have worked with?

You have that backwards. Schools don't want to share information on (say) behaviour because they are *very well aware* of the quality and they do not want to share that as a matter of course with parents, despite the fact that they are obliged to hand it over on request. Now you just have to figure out why they don't actually do something about that situation. My view is that the reasons lie in the fact that such data is the product of a process, it is not of itself the purpose of the process to produce that data.

I think I have responded to this, but what I am referring to is how schools wish to protect their data from reaching others, not with the focus of DPA and protecting the individual, but for the fact that the data in its current form (not format, but quality) can show the school in a bad light.

In some senses yes. What you don't really address is what would motivate them to make that change. The data supports an internal process, it was never intended to be published as a matter of course. The end product of that process that is shared with parents might be a letter or phone call and in preparing that, staff will refer to the recorded behaviour data as evidence.

 

Well now this is the million dollar question, and i'm glad you picked this up because it was cunningly placed there to ask these questions (I really do hope to realised that...): How can one have a master data source, and funnily enough I was having an excellent chat with @PhilNeal on just this topic. A view of the data is one thing, that is not connected as such to the master data source. However, where there are two sources that consider themselves to be the master data source (for instance, school is one, the social care department of the LA is another), you have to consider how can they work together. I'n my opinion (and @PhilNeal I am sure won't mind me saying broadly agrees with me here based on that conversation) they cannot by any technological means without an human interaction. I believe that the Data Exchange is not properly taking this into account, and actually schools & LAs, and more so the DfE, need to start thinking hard about this, lest the Data Exchange does not actually make any difference (or does, worse) to our lives.

Sure. Which is probably why their (Capita's) B2B product incorporated a 'distributed suspense' model at the LEA end where data conflicts were presented to the users to resolve at the point of use (rather than some monolithic central data cleaning process - although I understand that was an option too). Perhaps looking at authorities which make good use of that and authorities who don't, would help in understanding the problem - what works and what doesn't.

 

I'm not sure where you got master data MODEL from, I refer to source of data.

You refer several times to the concept of 'master data'.

 

"Data Exchange will be taking into account the apparent ‘mastery of data’; what is the master data source."

"Data Exchange will also be moving data (one way to begin with) on a more regular bases, meaning that the master data source; the school MIS and its data, needs to be in a more tip top condition"

" In a possible future, the actual master data source might be a single central record at the DfE"

 

See - three times, like the witches in Macbeth. I take that to mean your Data Exchange process is a model which features the concept of a "master data source". Am I wrong? If I am wrong, I'd suggest your article is, at best, not explaining things very well. IMO the concept of "master data" is ... misplaced. No one really cares about an abstract master authority, what people care about is the relevance of the data to them and the trust they can place in the data in order that it contributes to the quality and efficiency of their processes.

 

Therefore, I'm suggesting Data Exchange is trying to replace the vast collection numbers we currently have, and eventually to enable information that is shared currently by a multitude of means not just with DfE but other services internal and external of the school in a more convenient and joined up way, in a semi-automated fashion and in a timely manner. No none of this is new, but the way this is being done is new to the UK.

None of it is new, so why hasn't it succeeded before? Why are we still stuck with CBDS, ASC CTF or even csv via a report. I was talking about SIF well over ten years ago. It hasn't even succeeded in providing a means of data exchange between systems within a school, let alone different organisations. Our MIS talks to a number of other systems and none of them use SIF. Why? If it is advantageous why aren't suppliers flocking to use it?

 

I think if I was to sum up my 'concern' I would say "Those who cannot learn from history are condemned to repeat it".

 

@pcstru I'm happy to talk further, possibly offline and even a call if you want. I am very happy to explain & explore my opinions (!), but if you want to explore my competence to speak to this topic, can we please take that offline? Lets keep it professional because we always have good conversations on here otherwise you and I

 

To be honest... I'm just glad you read it!

 

I'm glad you wrote it and while my response is necessarily critical, it is not intended to be deeply personal (poking the odd bit of fun just keeps it from being too dry (It's not that I expect many people to read it but it's got to be a little bit fun to write!)). I'd rather discuss in here, openly because if the subject is relevant to people anywhere, it is here. If you don't want to do that fair enough, that's your call.

Edited May 10, 2014 by pcstru

[GREED]

Ok. I'm more than happy to discuss the points publicly, what I meant was if you want to discuss my ability to commentate on this subject, then take that offline...

 

I am glad you are critical because the intended purpose was to start conversations that I believe are necessary.

 

Where you mention SIF I would direct you to look at Australia where a mandate has applied to force all education data exchange to use SIF. Similar things are happening now and soon in states across America. Now. We are lagging behind these territories in that regard. I'm still concerned that there doesn't feel the need to have such a facility and we are quite happy to continue to silo information and use these old methods of data transfer. Things are moving forward. CBDS is being aligned with ISB standards. CTF will become obsolete with DX.

 

I want to clear up to I am not suggesting a model of any kind, rather very plainly a data source. The two are quite different things. And it is immaterial when we are agreed that there is potential not one true data source (as good as that would be in an idea world).

 

You are right too about suspense and my belief is we will never get away from that particularly in the LA. My fear too is that with DX not intending to have a suspense facility at DFE level, suspense issues will be exasperated...

 

You comment about data never intending to support external processes... I'm sorry but we have to start moving with the times!

 

Yes I conced that point in behaviour, I take that back schools are aware. I got that paragraph wrong. However if the data is not suitable or appropriate then something does need to be done for a) when a data request does come in and b) data exchange happens.

 

I'm not sure why there (not just you mind) seems to be this opinion that DX is a choice. Barring being cancelled at the next election this will happen. What I am trying to explore are the things that should and need to be done on the ground to properly manage this change in a gradual and acceptable way.

[GREED]

Also, to be clear, it isn't my Data Exchange... if it were things would be VERY VERY different!!

[dhicks]

CBDS is being aligned with ISB standards. CTF will become obsolete with DX.

 

I've got to implement some kind of link between our MIS (School Manager, based on Filemaker) and our VLE (Firefly). I intend to write code to extract data from Filemaker (if I understand correctly, Filemaker files can be directly published as XML data, so that part might actually be quite easy), store it in some kind of intermediate format, then send it to Firefly by whatever API they make available. What's my current best choice for an intermediate format - is it worth me writing a valid CFT file, a DX file (?), or something else?

 

And it is immaterial when we are agreed that there is potential not one true data source (as good as that would be in an idea world).

 

This strikes me as a similar kind of thing (multiple, distributed, equally valid, data sources) that the Git source code control system was designed to deal with. Distributed source code control systems are designed with computer programmers in mind, though - you might struggle to get the concept accross to your average school's data manager (or secretary, or teacher, or whoever else has to use the school MIS).

[GREED]

CTF is XML, as will any DX messages. The structure will be quite different based on what I have seen due to changes in the overall data structure being used at DX. There is nothing really available to work on, because it isn't finalised yet! You may be thinking to future proof if CTF is going but really this is years away. Your best bet is to identity what and how firefly needs the data and writing the XML maybe based on the CTF. What about automation of data flow? Happy to help further.

[dhicks]

What about automation of data flow?

 

The automation part I'm aiming to have handled by a file server that performs actions based on defined triggers, i.e. runs scripts when given files are updated. Updating a CTF (or DX, or whatever) file in a given location should trigger a script to place that data in the VLE - that CTF file could be produced periodically from a job that exports from the MIS, or might be updated by a user editing it.

[GREED]

Sounds plausible. Be careful that the triggers don't run up the processing on the database.

 

Your next challenge is the reconciliation of the data: do you want to completely over right the data each time of do you want to update what has changed only...

Edited May 10, 2014 by GREED

[pcstru]

Where you mention SIF I would direct you to look at Australia where a mandate has applied to force all education data exchange to use SIF. Similar things are happening now and soon in states across America. Now. We are lagging behind these territories in that regard.

Either would be, a bit of trek. I'm wary of anecdotes of far cities who's streets are paved with gold.

I'm still concerned that there doesn't feel the need to have such a facility and we are quite happy to continue to silo information and use these old methods of data transfer. Things are moving

forward. CBDS is being aligned with ISB standards. CTF will become obsolete with DX.

Sorry but I'm struggling to parse that. "I'm still concerned that there doesn't feel the need". ? !

 

ASC is a process linked to funding. If this new DX is to replace the existing CTF, what does that actually look like? Is this new monster nationwide DX project which will eat up time at 22,000 schools going to even save schools the wages of one admin assistant? It sounds very expensive, complex and onerous for all involved yet what actual value will it give anyone?

I want to clear up to I am not suggesting a model of any kind, rather very plainly a data source. The two are quite different things. And it is immaterial when we are agreed that there is potential not one true data source (as good as that would be in an idea world).

A 'model' is just a representation of something. I'm not sure what you think I must mean by the word, only that it is unlikely to be what I mean!

 

You have some sense of this new "Data Exchange" thing that is going to happen in a few years. When it happens it will be a thing, but the definition of the thing, is a 'model' of it - what it looks like. What problem is it even solving?

You are right too about suspense and my belief is we will never get away from that particularly in the LA. My fear too is that with DX not intending to have a suspense facility at DFE level, suspense issues will be exasperated...

 

You comment about data never intending to support external processes... I'm sorry but we have to start moving with the times!

Well OK, but so far you have failed to say what problem this DX is actually trying to solve. I'm not sure why for instance schools would send (say) behaviour data to the DfE, what the DfE would do with it or why?

Yes I conced that point in behaviour, I take that back schools are aware. I got that paragraph wrong. However if the data is not suitable or appropriate then something does need to be done for a) when a data request does come in and b) data exchange happens.

You seem to miss the point again. The data IS suitable and appropriate for the process it IS serving. Publishing it to parents would be a different purpose (parental engagement). If someone makes a data subject request, that is itself a different thing and they will just have to deal with what they get. It won't be pretty but what do they expect.

I'm not sure why there (not just you mind) seems to be this opinion that DX is a choice. Barring being cancelled at the next election this will happen.

I'm completely unaware of this new DX development! I suppose ASC will continue to be required but other than that, I don't see how schools will be compelled by some kind of external force to adopt substantially 'better' practices which seem to be assumed by you in order for the DX process to be viable (last 5 paragraphs in the blog). I simply don't see much value for schools in doing that.

[GREED]

Either would be, a bit of trek. I'm wary of anecdotes of far cities who's streets are paved with gold.

 

Not really anecdotal, just search for SIF in Oz and you can see it for yourself. I'm sorry I cannot provide an example from the UK which is what you are looking for because, well, we have not done it hear before. Doesn't mean we shouldn't try.

ASC is a process linked to funding. If this new DX is to replace the existing CTF, what does that actually look like?

 

Realistically, no one know yet. It is still being devised. I would provide the concepts but I fear without the hard proof this will cause further angst.

 

Is this new monster nationwide DX project which will eat up time at 22,000 schools going to even save schools the wages of one admin assistant? It sounds very expensive, complex and onerous for all involved yet what actual value will it give anyone?

Probably not, are you looking to lose someone from your staff then?

The system will replace all the census and returns from the school which must save hours and days over the year, each year. It should increase completeness and sameness of data across institutions and authorities. Seeing as this is being implemented directly into the MIS by suppliers, and we ignore any passing on of development costs to schools by said suppliers for a moment, is should not cost schools extra. When the process is up and running and embedded, it should save time, and as the processes are in the background of the systems, it will not be complex at all for users, or onerous (much less onerous than the current census processes). It will be a culture change, yes...

 

A 'model' is just a representation of something. I'm not sure what you think I must mean by the word, only that it is unlikely to be what I mean!

 

OK. Well I am simply referring to a database that is considered to have the 'master' data in that others feed off, seems a bit of a mute point as we have agreed and discussed that this is questionable anyway.

 

What problem is it even solving?

 

Aside from all the above mentioned things it will do, long term is will give a much more connected education system, shorter term it is replacing and updating processes that are long out of date, as DfE replaces their ageing data warehouse.

 

I'm not sure why for instance schools would send (say) behaviour data to the DfE, what the DfE would do with it or why?

 

Behaviour was used as an example only in the context of the protectionism of data by schools. Nothing to do with sending to DfE. The only thing for behaviour that is sent now and will continue to be in the future is exclusions.

 

I'm completely unaware of this new DX development! I suppose ASC will continue to be required but other than that, I don't see how schools will be compelled by some kind of external force to adopt substantially 'better' practices which seem to be assumed by you in order for the DX process to be viable (last 5 paragraphs in the blog). I simply don't see much value for schools in doing that.

 

I'm not surprised, at all. This has not been well managed with the public nor the MIS suppliers in my opinion, but that will come.

 

Next... schools won't have a choice. Currently they are mandated to do census and other returns, this will replace this and also mandate. The is not compelling. Whether there is value or not. It is just the same with the A2C project for exams. Soon you will have to send data via the new data structure and method, or you cannot manage exams. It is moving with the times. The MIS suppliers will be doing the work anyway so the change as mentioned before will be a cultural one. The external force will be the law and the DfE.

 

Given this fact that, barring a change after the election, this will happen, and you will have to adopt it, would it not be better to accept this and get on board with making the transition as painless as possible, and not procrastinating about the lack of benefits to schools? It would obviously be different if there was a choice in the matter. Don't get me wrong, these back and forths we are having are exactly what people need to see and so that when things do start moving the people who make the decisions are aware of the feeling on the ground (not sounding too self important there!). But I think I have clearly set out now what DX will try and solve (whether you feel they are issues or not is up to you), what the value is perceived to be and, to be fair, that value or not this is going to happen. I don't really want to go round and round on these points personally, if you don't feel satisfied by the responses then lets just leave it there and agree to disagree.

 

PS still feeling a certain amount of personal animosity in your responses. I am not responsible for the project I am just commentating on it. Assumptions are personal opinion (based on expertise and experience), but in general the benefits and value are ones laid out by DfE, not me! DfE has a whole book on what this will do and solve for the market backed up by mountains of research, if you really want to (fall asleep) know more.

[penfold_99]

Either would be, a bit of trek. I'm wary of anecdotes of far cities who's streets are paved with gold.

 

There are projects in the UK that are similar to DX but on a smaller scale (ie LA wide) that are using SIF. In Birmingham they use i to get up to date information regarding admissions number, this has a benefit for the LA admission team, the other is Warwickshire, they use it to push information about FSM eligibility directly into a school's MIS, the process is full automated, a parent can apply for FSM online (no stigma of having to get a form from the school or provide proof) this is then verified with DWP that a parent is eligible and if they are the information is pushed to the school, the benefit to the school is the pupil can have access to FSM within a day of application and the time they had to previously allocated to maintaining this information has been removed and can be allocated elsewhere.

 

ASC is a process linked to funding. If this new DX is to replace the existing CTF, what does that actually look like? Is this new monster nationwide DX project which will eat up time at 22,000 schools going to even save schools the wages of one admin assistant? It sounds very expensive, complex and onerous for all involved yet what actual value will it give anyone?

 

Well OK, but so far you have failed to say what problem this DX is actually trying to solve. I'm not sure why for instance schools would send (say) behaviour data to the DfE, what the DfE would do with it or why?

 

The majority of the benefit of the DX project will be at the DfE as they are the one footing to bill for it. One of their primary drivers are to increase the validity of the data, as the information used by different department (LRS etc) is anywhere between 6 to 18 months out of date, they are also wanted to move away from aggregated data to raw data, this is to reduce the number changes MIS providers are required to make when new questions are asked.

 

From a schools perspective DX opens up indirect benefits, for example data will be able to follow a student. Currently if a student moves school mid year the new school might be sent a CTF, which may or may not contain all information, attendance and assessment are in the majority of cases are not included. The new school need to spend weeks assessing the student to ascertain what level they are at and which groups they should be in, If data followed the student, once they had been entered in the MIS it could be back filled via DX reducing the time teachers would need to assess them (as they would have a baseline to work with) and the admin time to enter all this information would be removed as well.

 

On a side note, behaviour data is always a bad example to use as currently there isn't a national standard on how it should be recorded plus is very subjective and heavily influenced by the context of the school and local community.

[matt40k]

Do you know what my over simplistic view of the data exchange project is @pcstru - it's the DfE standardising Capita B2B process. Maybe the "census" would be produced from the data held higher up the stream, and the school would approve it. I don't see that approval ever disappearing, I mean we're human, we make mistakes. You need the ability to hey, that's not right, hang on a min, I need to correct that. Otherwise the data will be total rubbish. The aspect is the governments places to bring everything under one banner. It makes sense, how accurate would you say the EduBase data is for example? It's looking at what data is pulled and going hey, we ask for x multiple times for tasks a, b, c. Lets get it once, then reuse.

 

Regards to SIF, yer, it is a bit of a buzz word people like to throw around, after speak with @GREED, my basic understand is it's a protocol. The big I have a problem with it is the endpoints, which really isn't SIF, SIF is just how the data is "formatted" in transmission, the endpoints is the bit that will deal with the data and that's the bit I think the problems will occur. How you deal with the suspense.

 

If we look at @penfold_99 excellent example of FSM, this is a brilliant use of the system. As a parent I can apply online, I don't need to read some overcomplicated form about if I am eligible or not, I just apply and if I am, my school gets extra money if not, oh well, I get a letter in the post (bizarre that they have to write a physical letter to you for a digital process). I think what the DfE want is a standard rather then a proprietary solution so [schools] aren't discriminated by the [LA] choice of system.

 

Regards to not sharing, in the case of safeguarding - specifically when a person has a doubt or a concern. How do you deal with that? As a parent I would hope they would address their concerns with me, but if they still had doubts, I respect they would want to have a report expressing those doubts so that if other occurrences happen, they have a strong case and not some stories that start "well, I remember...". My only hope is that these are shared, I don't want them to sit in a filling cabinet only to be wheeled out post-mortem as a form of arse covering. I want it to be used proactively. In Suffolk there is a project called Family Focus that is joining services together to build a bigger picture, so health, police and social care data is combined to give a wider view of the whole family. Data needs to be shared, it needs to be used. Otherwise it's useless. People can't make good decisions without good information and that means data and anything, be it SIF or otherwise that makes it easier to get the data is worthwhile, especially if it helps prevent vendor lock-in.

[pcstru]

There are projects in the UK that are similar to DX but on a smaller scale (ie LA wide) that are using SIF. In Birmingham they use i to get up to date information regarding admissions number, this has a benefit for the LA admission team, the other is Warwickshire, they use it to push information about FSM eligibility directly into a school's MIS, the process is full automated, a parent can apply for FSM online (no stigma of having to get a form from the school or provide proof) this is then verified with DWP that a parent is eligible and if they are the information is pushed to the school, the benefit to the school is the pupil can have access to FSM within a day of application and the time they had to previously allocated to maintaining this information has been removed and can be allocated elsewhere.

What I see as very different in that, is that functional process is driving the system. It is beneficial to schools and the LEA, they both get something out of it, accuracy, timeliness, efficiency. The information they are dealing with is minimised to the information required by that process. It's very, very focused.

 

This is not apparent in the DX we are speaking of here. There may be advantages down the road IF it could be made to work, but that will be too late. You need a compelling case right from the start and particularly missing from the DX case is the advantage for schools. Not some mythical potential to improve efficiency in a few years when it all works, but right from the outset.

The majority of the benefit of the DX project will be at the DfE as they are the one footing to bill for it.

How? Will they be putting money into schools to pay for the set-up and maintenance? Will they pay for the staff time to investigate and correct problems when the system goes wrong? Will they pay a retainer to suppliers to build the software?

 

We were asked to put in a feed for the LEA, they would suck data out of our system for their use. We refused on my recommendation. There was simply nothing in it for us except the work to make it work - it would cost us time and effort to set up and every time something went wrong. There was no disadvantage to the school in refusing so why would we?

 

One of their primary drivers are to increase the validity of the data, as the information used by different department (LRS etc) is anywhere between 6 to 18 months out of date, they are also wanted to move away from aggregated data to raw data, this is to reduce the number changes MIS providers are required to make when new questions are asked.

IMO there is a misunderstanding somewhere. ASC for instance is more a data quality exercise than a data transfer exercise. A lot of effort goes in at the school end to make sure the data is a true reflection of the school because funding is predicated on the result. Similarly a lot of effort goes in by the DfE and software suppliers to specify and produce valid, accurate data. It's just about doable as a one off exercise every so often. Keeping that quality all the time - forget it.

 

The only way to change that, would be to change the process. So the DfE would have to change to a system where they say funding will be allocated on the aggregate picture across the year - at which point sensible people will ask "what exactly does that mean", because they will want to know the EXACT basis on which funding will be calculated, not some vague hand waving. None of that is really technical.

 

For the LRS in your example, their data is 6-18 months out of date not because it's not possible to transfer data faster, but because assuring the quality of that data cannot be done any faster. Low quality data is sometimes worse than no data but perhaps even worse than that is data where the quality is just unknown - it might be right but it might be horribly, horribly wrong.

From a schools perspective DX opens up indirect benefits, for example data will be able to follow a student. Currently if a student moves school mid year the new school might be sent a CTF, which may or may not contain all information, attendance and assessment are in the majority of cases are not included. The new school need to spend weeks assessing the student to ascertain what level they are at and which groups they should be in, If data followed the student, once they had been entered in the MIS it could be back filled via DX reducing the time teachers would need to assess them (as they would have a baseline to work with) and the admin time to enter all this information would be removed as well.

Other than independently assessed 'examinations', do you think schools will rely on the quality of assessment data coming from another school? And why would attendance data from another school matter to them? There was a previous DfE site for exchanging CTF files - what happened to that (why didn't it really work and why are CTF files emailed around between schools instead). I doubt the answer is because the technology is not capable of doing the job.

On a side note, behaviour data is always a bad example to use as currently there isn't a national standard on how it should be recorded plus is very subjective and heavily influenced by the context of the school and local community.

 

Sure. In fact a lot of data in a school will a bad example and useless to another school because the processes and standards are different. Similar but different.

 

I'm cynical of this because like many projects before it, I'd vision a bunch of mostly technical people sitting around talking about XML schema, SOAP/RPC type protocols, resolution of data conflicts etc. I've been in those meetings. There is very little appreciation that the main problem is procedural and the impact on schools to make it work would be huge. Even if there is appreciation of that, the people who can do something about it are not in the room or actually involved.

 

Let's take the bad example - Imagine standardising practices to improve behaviour in schools. Imagine a government trying to impose a national system of behaviour management. I mean the advantages would be potentially huge - imagine all those data managers who would never have to touch the reporting systems because suppliers would supply the same report to everyone. And because the process and procedure would be exactly the same at every school, no one would actually need a different report. This seems to me little different from the nirvana promised by DX - huge potential future advantages being sold, little real thought as to the cost and effort that would be required to achieve it.

Edited May 11, 2014 by pcstru

[pcstru]

Do you know what my over simplistic view of the data exchange project is @pcstru - it's the DfE standardising Capita B2B process. Maybe the "census" would be produced from the data held higher up the stream, and the school would approve it. I don't see that approval ever disappearing, I mean we're human, we make mistakes. You need the ability to hey, that's not right, hang on a min, I need to correct that. Otherwise the data will be total rubbish. The aspect is the governments places to bring everything under one banner. It makes sense, how accurate would you say the EduBase data is for example?

I don't think it is possible to look data at an organisational level and say how accurate the data is in any meaningful sense. It is only possible to take a some given data and then attempt to say how accurate that actual data is. Sometimes, even trivial 'data' questions are actually very difficult. "What temperature is the room?" looks like a simple question to which 19deg C would look like a simple, accurate answer. The reality is more like 19deg C +/- 5 deg C. In other words, Data quality isn't simply about accuracy, it is quantification of error. typically you need to have quite a lot of meta data to do that quantification, what size was the volume, how accurate was the instrument, where was the instrument place in relation to heat sources / drafts, how well mixed was the air etc. And that's of a simple single piece of data.

 

I'd strongly agree with you that ASC which is used as a basis for school funding, will always be subject to sign off by the school and that itself suggests that it has to be a snapshot. There is no benefit to the ASC process from continual data exchange.

 

Regards to not sharing, in the case of safeguarding - specifically when a person has a doubt or a concern. How do you deal with that? As a parent I would hope they would address their concerns with me, but if they still had doubts, I respect they would want to have a report expressing those doubts so that if other occurrences happen, they have a strong case and not some stories that start "well, I remember...". My only hope is that these are shared, I don't want them to sit in a filling cabinet only to be wheeled out post-mortem as a form of arse covering. I want it to be used proactively. In Suffolk there is a project called Family Focus that is joining services together to build a bigger picture, so health, police and social care data is combined to give a wider view of the whole family. Data needs to be shared, it needs to be used. Otherwise it's useless. People can't make good decisions without good information and that means data and anything, be it SIF or otherwise that makes it easier to get the data is worthwhile, especially if it helps prevent vendor lock-in.

 

There was a project started in ~2003 (+/- a few years, I'm getting too old to remember my own name in the mornings, let alone stuff from10+ years ago!) to do information sharing between departments and systems - it was in response to the case of Victoria Climbié. As I recall, various specifications were drawn up and software development was even done by various companies so that data could be exchanged between the systems. It was more SIF like than anything before or possibly since, which is another opportunity for a "why didn't that actually work" question?

[PhilNeal]

That project was stopped for political reasons.

[pcstru]

That project was stopped for political reasons.

That doesn't tell anyone anything much. What do you actually mean by that? It was certainly stopped by politicians but was it was somehow stopped when otherwise, all elements of the business case remained viable? Or was it stopped because it was seen as another government IT project which was lacking proper project control and was likely if it continued, to fail to realise the benefits of the business case? Surely politicians don't stop projects and merely claim "oh, well Prime Minister, we, err, stopped it for, ummm, 'political reasons' ", they at least lay claim to some rationale.

 

Perhaps a little more useful would be to say in your opinion as a supplier, do you think it was stopped for good reasons or bad? Was it a right decision or a wrong one? Had it gone ahead would it have worked and would it have been good value?

[GREED]

You will likely be very surprised then... projects are stopped at a moments notice often when a new government comes in because it doesn't fall into their political agenda... shall we all play 'name that stopped project'?

[elsiegee40]

You will likely be very surprised then... projects are stopped at a moments notice often when a new government comes in because it doesn't fall into their political agenda... shall we all play 'name that stopped project'?

 

I'll play ... PFI and raise it with BSF

[GREED]

I'll see your BSF and ill raise your the National ID Card scheme

[PhilNeal]

Political with a capital "p". http://www.libdemvoice.org/321and-contactpoint-is-gone-20603.html

[pcstru]

Political with a capital "p". 3..2..1…and ContactPoint is gone

 

Thank you. From your link, the actual reasons given were "intrusiveness, cost and problems with ContactPoint".

 

I'd hope it would be fair to say that whether you agree or disagree with that assessment, it is unlikely that there was no evidence on which those opinions were based. Only the first on the list is almost entirely Political as a value judgement and even then the politicians would have been looking closely at an actual metric, how many votes might that mean. The cost was significant enough that it was even on the radar of Senior politicians outside of DfE, and there were ... genuine problems both at a conceptual level [*1], at operational levels, and technically.

 

[*1] Conceptually - I mean the whole idea of putting "widely sharing" into the same sentence as "extremely sensitive confidential" is only a good one if the sentence also contains some sense of "fail".

[PhillipHamlyn]

From the technical perspective of a supplier, the key difference between the DX project and the various school census it replaces is the requirement to feed data back into schools automatically from the DfE data warehouse; a data flow which does not exist outside of CTF, and then only for a limited data set encompassed by the CTF.

 

A technican may argue that submitting changes from the school through to the DfE via a more immediate protocol than Termly/ad-hoc would just make the data held and analysed at the centre more timely with the same level of accuracy; but perhaps a realist would admit that the "operational gap" between the entering the data which contributes to the census, and the actual validation it and submission of it, reprepsents a good quality assurance process, which will be eliminated or reduced if the data were submitted immediately (or near real time). The DX protocol as I currently understand it does not envisage schools blocking or suspending the outward flow of data to the DfE, and the ambition is to get near-real-time data from the school eliminating all termly or scheduled data collections.

 

The most problematic issue which arose when we designed the B2B protocol/product that led to us implementing some quite comprehensive suspense rules and protocols, was the issue of data owership. A good example might be

 

1) If a child tells their school, via a letter from a parent, that they have moved house; the school enters the data, thereby updating the LA/Dfe (potentially).

2) If the LA is informed the child has moved house through some other channel, and update the school database via a data synch product (B2B, DX etc); but the school staff are not positively informed (i.e. not in the update loop), then who is the data owner ? and who is the guarantor of data quality ?

 

I'm sure there might be specific legal definitions in these cases, but as I understand it it is the school who registers themselves as the data controllers currently, so are responsible for the accuracy of the data they hold ?

 

Our answer to this was to introduce in our product suspense capabilities allowing recipients of data to "hold" updates in suspense until they had eyeballed them and either rejected or accepted them. We believe that this same process will be potentially required within the school with the rollout of DX, and this will in fact increase the data administration tasks done by a school, depending on the volume of changes received through the DX channel. It may well be that these two facilities of DX (submission from school to Dfe and from DfE to school) are implemented in seperate phases, but at present I don't have any information to indicate that this will be so.

 

To comment on another line of conversation in this thread and based on my understanding of the goals, scope and business case of the DX project; it will explicitly not address data transfer school->school or data transfers within a school. Its a school->Dfe->school data transfer protocol (that might happen to make CTF obsolete). This may have changed since my brief involvement in reviewing the technical approach, but I believe its unlikely.

 

Phillip Hamlyn

[GREED]

Philip

 

This will replace CTF and school to school transfers as we know them today. It will 99.9% certainly go school dfe school as you suggest, but school transfer is very much on the medium term plans.

[PhillipHamlyn]

GREED,

 

Understood, but the route will be School A ->Dfe-> School B ? So not direct school to school ? Correct me if I've got out of date info.

 

Phillip

[GREED]

Yes you are right Phillip that is the current thinking, via dfe (well, the data exchange, not the data warehouse). During discussions it was deemed unfeasible to have school endpoints, all 24000 of them, being aware of all other 24000 schools to allow direct communication.

[pcstru]

A technican may argue that submitting changes from the school through to the DfE via a more immediate protocol than Termly/ad-hoc would just make the data held and analysed at the centre more timely with the same level of accuracy; but perhaps a realist would admit that the "operational gap" between the entering the data which contributes to the census, and the actual validation it and submission of it, reprepsents a good quality assurance process, which will be eliminated or reduced if the data were submitted immediately (or near real time). The DX protocol as I currently understand it does not envisage schools blocking or suspending the outward flow of data to the DfE, and the ambition is to get near-real-time data from the school eliminating all termly or scheduled data collections.

Extremely helpful detail. Thank you. So essentially a one way transfer?

I'm sure there might be specific legal definitions in these cases, but as I understand it it is the school who registers themselves as the data controllers currently, so are responsible for the accuracy of the data they hold ?

Interesting question. "Responsible for the accuracy" with respect to what? When we carry out an action or a third party acts on our behalf, using data we have supplied, then we are liable for our actions because we have a duty of care. If we supply data to the DfE and the DfE acts on on that data in some way independently of the school, does our duty of care apply? Do the DfE have a get out clause under DPA anyway? What if we accept information from the DfE - are we acting as suppliers for them, under their registration or under our own? Does it matter in terms of a school "upping it's game" - they presumably are already motivated to hold data of a quality in a way that appropriately manages the risk. Why then would DX change data quality?

Our answer to this was to introduce in our product suspense capabilities allowing recipients of data to "hold" updates in suspense until they had eyeballed them and either rejected or accepted them. We believe that this same process will be potentially required within the school with the rollout of DX, and this will in fact increase the data administration tasks done by a school, depending on the volume of changes received through the DX channel. It may well be that these two facilities of DX (submission from school to Dfe and from DfE to school) are implemented in seperate phases, but at present I don't have any information to indicate that this will be so.

So more work and not a one way process?

 

To comment on another line of conversation in this thread and based on my understanding of the goals, scope and business case of the DX project; it will explicitly not address data transfer school->school or data transfers within a school. Its a school->Dfe->school data transfer protocol (that might happen to make CTF obsolete). This may have changed since my brief involvement in reviewing the technical approach, but I believe its unlikely.

 

Many thanks for chipping in. I guess what I'm now clearly very unclear about is the detail on a) the two way stuff. What data will the DfE be sending back and b) what motivates a school to achieve and maintain a 'higher' data quality at times other than when "ASC snapshots" will be in play and what makes it possible, other than data quality to make those snapshots more frequent.

 

(Sorry for making up "ASC snapshot" as term, hopefully you know what I mean - what exactly goes into the funding model).

Edited May 11, 2014 by pcstru