Never ending data... but what's it all worth?

[LosOjos]

Now don't get me wrong, I'm all for data; it's my bread and butter! But how much data is too much data?

 

Every time I manage to get a good foot hold here and get my systems to a point where I can comfortably meet deadlines, SLT throw in another request for a load of extra data that must be produced and distributed with all the other data for all staff. Today for instance, not content with every teacher receiving print outs of every class they teach with all their students key data, targets and attainment, they now want photos adding to it (I've pointed out dozens of times what a massive data protection risk all this printing is, not to mention the cost and the fact I make it all available digitally and securely, but I feel it's falling on deaf ears)

 

There are certain members of our staff who quiz me on data, who genuinely sit down and look at their data and want to know more and I'm happy to help them. But they are a very small minority. I estimate that 90% of them simply shove the print outs in their folders and forget about them.

 

I must print out well over 3,000 sheets of A4, in colour, every term. 3,000! It terrifies me how much of that data could be sat in bins, drawers, pubs, buses etc. at any given moment in time!

 

I really do have to question the value of all this data being produced when so few seem to take notice of it.

 

Do others face these kinds of challenges, or is my school a particularly bad one? I've never worked in another school so sometimes I don't know whether it's just me!

[NormalDistribution]

There's a mantra I oft repeat to myself and occasionally others which is "Data, Data Everywhere - no time to stop and think".

 

Of course on the bare bones of things, we are a support service. We provide data to support teaching and learning, however this does not preclude us from stopping unnecessary wastes of time such as you describe.

 

From your description of yourself in the other thread, you are a developer at heart, and really this fits right in with how you need help the school to work smarter with it's data. I see myself as an educationalist with a modicum of development skills and it is when I manage to marry these two elements together that I get the best results.

 

Having lots of data is not what it is about, we are not called Data Providers, we are Data Managers. What really counts is having the right amount of managed data in order to evaluate and make strategic decisions confidently and successfully.

 

The aim of someone like yourself with development talent is to put these analysis tools in the hands of teachers. I say "tools" because it should be more than just a regurgitation of what they already know. It should have the ability to tell them something that they can then use to impact learning.

 

However you can not go alone - you are not an island (!) - whatever senior leaders want the staff to do with the data should be made clear - any evaluation exercises they need to complete should be focussed in a way that harmonises the data provided with actions that need to occur. In many cases it is as important to realign what senior leadship are looking for and the way they present that message as it is to provide the data itself.

 

Back to your photographs of students - why? Aren't these already in SIMS?

 

There is a balance between being too helpful or too obtusive. But if adding something like photos to your work - adds no value - or less value that the effort it takes to do it then they should be encouraged to reconsider or use a different source.

[LosOjos]

I whole heartedly agree with you!

 

Which is why I couldn't help but be lost for words, only managing a "why?" when asked to add photos to all these sheets of data, which not only creates a load of work in setting up an entirely new way of producing the data (as you can't add photos to marksheets in SIMS) and will vastly increase the cost and volume of printing, but I just don't see the benefit to be gained from it. SLT's argument is that it's about "putting faces to names," but I don't buy it. It's about making the print outs look pretty IMHO!

 

Of course, as you say, we are essentially service providers and so if this is what SLT wants, this is what I will produce. I suppose I just wish I had more control over the school's use of data - I do feel as though I'm being used as a tool to churn out endless reams of data and am never actually involved in any of the decision making.

[NormalDistribution]

I would just say you need to investigate whether it is technically possible...

[LosOjos]

Yes, although I'm sure I could do it, it is so tempting to say it can't be done.

 

In reality, I won't do that, but I will try to make them give me a damn good reason why before I do it. It's not going to be an easy task that's for sure.

[NormalDistribution]

It may be worth waiting to see what the new SIMS brings... I am under the impression they are working on a major upgrade/new product, possibly from the Summer.

[BKGarry]

We do have similar sheet for data, each subject area has a Group Information sheet, these where created at they wanted all of this information at the begining of each marksheet and they where getting to big. They love the convenience of them being in SIMS in assessment manager, as they appear as a tab at the top of their Lesson Monitor Registers, if they want photos they use the class photo button in the same register, also it being in SIMS they can click on the students name and go to the sheet.

 

I have included a screenshot (without the name column) if you have any questions about any aspect, please ask, as i know OFSTED really did like these a couple of months ago, as teachers have the latest information on their SIMS screen and they can print them out and provide the info to a peer inspector or the OFSTED inspector and they was all in the same format.

 

http://ntc.kent.sch.uk/new/2013-02-07_1707.png

[skunk]

@BKGarry like looking at a carbon copy of our "Progress" marksheets. If I were Apple I would sue..........

 

@LosOjos Probably teaching you to suck eggs, but I get in front of staff as often as possible and show them how to access class lists from the reports, photos from the class lists, photos from their registers, filter their own marksheets, etc. I do not know if you have something like Impero, but it is great for showing staff how to get the data themselves. If anyone tries to get me to print data out I put it on a shared drive and email the link (unless it is for discussions between teachers and students when I occasionally relent). I have started adding commentary to group data analysis to perhaps point to areas that I believe are a concern or show positive or negative trends. Although writing "The data are showing that those who attend less do less well" fifteen times a year is getting wearisome.

 

Still I do far too much printing (i.e. any) my opinion. We have had decent screens for decades, why do we still rely on paper so much?

[number34]

That table from SIMS can probably be duplicated in most secondary schools and I would agree that sometimes I add aspects and columns which seem to have no relevance at all, are hardly looked at and not understood by the vast majority of staff. My main worry is that hardly any of the people have any idea of how to interpret even the most basic statistics that are generated from the data and use them in the wrong way. However they are only doing what OFSTED and the DFE do all the time so perhaps it does not matter, especially as the various exam and curriculum sytems are changed so frequently that no valid comparisons of any sort may be made.

On the subject of pretty mark sheets keeping them happy I find that lots of colour coding satisfies my SLT!

[LosOjos]

@BKGarry yeah that's almost identcial to ours!

 

@skunk yeah I do the same, put all these marksheets on our VLE, sorted by staff initials and separately by subject, but they always come back with the "we don't have colour printers" line. We also use SISRA for online analysis which I can see from our logs is used by 10% of the staff at best.

 

The real problem is that it's SLT constantly pushing for all this stuff to be printed, despite my best efforts they simply will not back down on it. I've tried the data protection route, even naming and shaming staff who had admitted to losing their print outs (nothing was done about it) and I've tried the money route which also seems to be ignored (despite the fact that we're about to face a third round of redundancies due to our massive budget deficit).

 

Frankly, I think I'm on a sinking ship!

[NormalDistribution]

Excuse my ignorance - how do I link to a screenshot from my PC like BKGarry - not from an URL? I've got the screenshot sat here as a .jpg

 

 

To clarify - when I click the add picture it only gives me the option to add from URl but Iif I go to a forum I can choose between Computer or URL.

[Ecclesbury]

You know who I blame - OfSted.

 

When they do their inspections everything has to be on paper! I heard stories of outstanding teacher being given 'satisfactory' purely because they didn't have a bit of paper with some grades on to give to the inspector, regardless of the fact that the teacher could bring the grades up on the computer in the same classroom as the inspector. At our last inspection I had to come to work on a Saturday, not to prepare any extra data, just to print the reams and reams of stuff that we already had.

 

I also believe that they are the single biggest contributing factor to inefficient data systems and the data saturation that LosOjos is talking about. I am currently looking at how to effectively monitor progress at KS3. The biggest sticking point is that measuring genuine progress in foundation subjects (especially creative arts such as Drama, Music and Art) doesn't tally with what teachers and departments will be judged on by OfSted (i.e. Levels of progress from an average KS2 level.) So we end up having to produce 2 independant sets of analysis at different time. One to give the picture that OfSted will see, and the staff will be judged on, and another set which foundation staff can use to justify the ugly looknig OfSted data ("see, they are REALLY making progress in music")

 

Throw in ever shifting goal posts and all the (changing) sub-groups that staff are expected to monitor, leads to this problem that either staff are bombarded with an obscene amount of data, or they can not put there hand on a specific bit of data when they need it (and come running to us for it!)

 

I agree with those who have said that the key is organising the data in the most simple and efficient way possible (The MIS is crucial for this) and then spending more time/effort in training staff how to access the bits they need, when they need them, rather than just churning out summary tables as and when they are needed.

 

One thing that our staff have responded well to, is when I produce excel summary sheets at key assessment points I include my own dynamic graphs. This way they get a nice visualisation of, say, how the sub groups compare for A*C. They then have a simple drop down box which automatically updates the graph. Just a small example of where a lot of data is provided to staff but in a way which is not overwhelming and can be interrogated easily.

 

There's only one thing worse than too much data, and that's too much waffle about data! So apologies for this post!

[LosOjos]

@Ecclesbury some very good points there, I think you may have highlighted a key issue here - Ofsted. It ought to be illegal to mark teachers down for not having print outs, as print outs create an unneccessary data protection risk (in fact, it's arguable that print outs are in breach of data protection, but I won't go there!)

 

You've also raised a very valid point - organisation. I think data analysis and assessment is an area that is greatly lacking in the education software realm. There are dozens of tools available but I'm yet to find one that fulfills all my needs, and talking to the rest of you we all have very similar situations involving a myriad of systems being run for different data sets, which in turn causes data-illiterate staff to simply hold up their hands and give up!

 

I have considered developing my own system in house that would provide every bit of data that staff could possibly want and turning SIMS in to purely a data entry system, which it essentially is already, AM7 leaves much to be desired in terms of valuable analysis and involves far too many workarounds to produce the data we need. The only problem is of course finding time to develop this system in between printing all this bloody data!

 

Perhaps a summer project to work on.

[LosOjos]

@NormalDistribution - to attach an image to a post, click 'Go Advanced' and you'll have an option to add attachments

[NormalDistribution]

Something must be wrong (possibly with me) - when I go to the "insert image" icon from this thread I only get - Add image from URL.

 

However if I go to a forum thread I can switch between "From URL" and "From Computer"

[skunk]

OfSTED are a nightmare, we have even had one inspector look at our analysis, say, "There's too much red, not enough green, this is a bad system". We just changed the colour meanings in the system so that shades of green appeared. Did it help or hinder the students, no, but OfSTED loved the same system from then on. But they are given their remit from the Govt, so we can all bring the blame back to Mr Gove and his pals!

 

In terms of MIS, we are looking at ASPEN next month to see what it can do for us in place of SIMS, for a whole host of reasons, not least data analysis and management. And Arbour have come up with a neat marketing trick by analysing all schools raiseonline data for free which looks very nice.

 

Capita have sold as many licences as they are ever likely to for the base system (although the academies conversions must have had them rubbing their hands with glee) and their sales guys clamor for further add-ons to sell, gotta get those bonuses. There is little drive to improve the core system as it will not bring in a great deal of sales or additional revenue so development effort is spent elsewhere. Just look at the FFT tool for a relatively pointless add on which brings in additional annual income for Capita.

 

Just a cynical ancient user of SIMS. But it could be so much more and it frustrates the hell out of me.

[NormalDistribution]

So anyway.. I looked back at BKGarrys picture and decided that it was from an URL so I have uploaded my picture. Hurrah!

 

Here I have exported the data from SIMS to create a class output for teaching staff - much the same as what you are talking about I think.

 

http://i49.tinypic.com/33eslko.jpg

[number34]

That is a vey nice summary for a class, how often is it updated to reflect changes and how easy is it to do?

[LosOjos]

@NormalDistribution - yeah that's where we're headed. We don't have graphs just yet (just colour coded tables at the moment), but I'd like to add those in soon. SLT haven't actually mentioned graphs, but I'm sure if I put them in they'll love them.

 

My blue sky idea is to essentially build my own analysis system and put it on the intranet. Ideally, it'd privde a central source for all data analysis, in a format not dissimilar to 4matrix (which I'm desperate to try but having a hard time convincing SLT) and SISRA, with the added bonus of being my own system that I can add, change and update as and when I please.

 

If it's easy enough to use, I hope it'll help me cull the amount of printing too... a big part of that problem (aside from the unavoidable Ofsted) is that currently staff "don't know where to look". They do really, I tell them often enough, but the moment they have to look in more than one place many give up, hence my hope to develop this system some day...

 

Who knows, I might even be able to sell it to you lot in a few years

[Pico]

@NormalDistribution Thanks for sharing your screenshot. I've been thinking about doing a dashboard design for SLT for high-level breakdowns, but that's a nice high-level summary of a set.

[Ecclesbury]

Very nice summary sheet that @NormalDistribution! Is it done in Excel?

 

It's given me some great ideas. I particularly like the Key Points.

[vikpaw]

Very nice data view.

 

As for Ofsted, when i was in the UK, we were told it was okay to have data online, so long as you could supply it to them when asked. So a lot of data was stored on the intranet or shared areas. Things may have changed though, or maybe some inspectors are not as IT literate as they should be.

 

I agree with you all there is way to much data and not enough information. Maybe we should change our titles to Information Managers, because i believe that is the key, turning the oodles of data into meaningful info.

 

I'm interested to see how 4Matrix works and how live it is. A good way to try to wean people off paper is to say that online data is way more up to date. If you can convince staff and management that as soon as they change one kids grade, it will immediately affect their stats, the teacher stats and the class stats, you would hope it would convince them it's much more efficient to be online.

 

That's why i like Discover, though haven't implemented it here, because it's just not needed, and the overhead on the systems would just kill things. The principle it works on is very nice. SQL has built in Business Intelligence tools that are supposed to do similar from SQL 2012 onwards, but the key is providing an underlying scheme and showing it where the data is stored. This is where we're over a barrel with SIMS, it's so hard to access the data.

 

If you can build a system where your SIMS data is copied to another SQL server, or even just accessed via read only views, you'd be well on your way to a nifty analysis system.

 

I was equally impressed with Arbor and Aspen at the BETT show recently, and loved the even better marketing gimmick (arbor) of saying they will rip your SIMS data out and into the cloud using a Raspberry Pi , plus you get a demo of their system, in the cloud with your data in it. That is cool.

 

I do hope that SIMS will catch up and go cloud, but it's just seems too difficult. It would be a total port to a more logical interlinked system, so could be years away, though i reckon they are working on it in secret, but won't announce anything for a while.

[NormalDistribution]

@Number34, @Ecclesbury - Yes that screenshot is from excel. I create it through the running a report in SIMS which pulls out all the required data into excel. I then drop that data into the student sheet that sits behind the screenshot I have provided.

 

I then have to update the dropdown lists to reflect any new classes that have been created (usually intervention groups). At this point I usually have to look up a couple of students who have joined that I don't have prior attainment for. From start to finish it takes about half a day to populate and check it is working.

 

Obviously the downside is that it is static. I update it a couple of times at the start of september - as classes settle down, then again at the start of each term. I lock it down so staff can't break it - plus it is in a read-only location.

 

As I might have said before I have no problem with excel as long as it is being created by someone who has a clue what they are doing, it is when non-data specialists get tangled up in excel that problems arise. I am by no means an excel expert but competent enough in designing it to be user friendly.

[Pico]

@Vikpaw I agree that we are in many respects Information Managers. Yes, we need to ensure the integrity of the data and the soundness of data collection methods (within a set of constraints we don't always have control over!), but we also need to analyse and present that data to inform monitoring and decision-making.

 

Just to correct an impression you might have inadvertently created, the BIDS tools are not just available for SQL Server 2012. We started using Reporting Services when we were on SQL Server 2008. In fact, the reports you create can connect to different kinds of data sources, so you're not even restricted to Microsoft DBMSs. Having said that, if you can't readily access the database, as you seem to imply about SIMS, then it's no good.

 

Personally, I've found it to be an extremely powerful tool for two main reasons: you are working with live data and the reports can be interactive. On the downside, some of my SQL queries are horrendous. I've seen extremely simple MDX queries that do the same thing, but I'm not sure I can justify the overheads of working with Analysis Services.

 

I'm in two minds about whether the fact we're working with live data has helped with reducing the amount of printing. It has helped with some things, but not others. One issue is that when Form Tutors fill in comments on reports, they like to see an overview of what a student has had across the year so far. Presenting all this information on screen, alongside the form for filling in the comment, is tricky when you're working with a laptop. There just doesn't seem to be enough screen real estate to get everything displayed and it is sometimes hard to get a freeze panes effect in the report. I sympathise with some teachers when they prefer to work with a print out, even though I agree that it is costly and needs to be properly destroyed.

 

Anyone know of any good training courses for report design for on-screen display, because I know I'm running up against usability issues, but I have no particular knowledge of the area I'm also pretty sure this goes hand-in-glove with displaying data in a meaningful way. That's one reason why seeing alternative presentations, such as NormalDistribution's is so helpful.

[vikpaw]

Soz, i didn't realise what was possible with 2008. I just saw it at a Microsoft seminar recently (a local one!) and they were saying it's the new cool thing in 2012. Maybe it was just some of the 'new' features that were only for SQL 2012. It does work equally well with excel and access as sources, but i much prefer going straight into the DB. We are not allowed access to SIMS backend, not officially anyway, so it's a nightmare, and it's a double headache finding out how the tables all link together, add to that the fact they frequently changed things around.

 

With marksheets in SIMS, we use Assessment Manager, the grading module to do reports it's quite easy to show previous year's grades, as the comment entry box is a tiny Excel Cell like box. This pops open to a data entry window. I'm sure a lot of our staff would prefer paper copies, but most wont bother with teh complexity of exporting to excel, then fiddling with the protection and layout to get it printable. Luckily, i'm not asked to provide.

 

What exactly do you mean by report design for on-screen display, what are you doing at the mo'?