The Analysis of Assessment Data in your school

[Tyiell]

@Mathews - can't you use analysis and trend tracking tools in AM7?

Don't know about Matthews, but we have tried those, and while they are useful, we find that they don't provide the flexibility we need to satisfy the powers that be!!! Wish it did!!!

[garrysaddington]

Hi
We need software that can tell us:

Percentage A*-C in KS4 & 5 in each subject for Assessments, Targets & FFT or Alis
Percenatge 5+ & 6+ in Ks3 Subjects for Assessments, Targets & FFT

This data also needs further analysis to tell us how many of these are Girls,Boys,SEN,FSM,EAL & Ethnic Minorty Students.


Kelly

Do you mean like your MIS supplier should have already been doing, without making schools do the work. See the link below (from 2008).

Management Information Systems (MIS) and pupil tracking: Developers' guidance

What is the point of collecting all the data in a MIS to then have to export it somewhere else to do something useful with it. Why not have a spreadsheet open in the staffroom to collect the data and cut out the MIS? Then ask for a reduction in annual fees for your MIS.

[Tyiell]

That document seems to highlight what you should be doing without really going into how - most particularly it is not specific to Sims. I can't speak for CMIS, but SIMS does not currently have an in-built solution to provide the extensive data analysis we are asked to provide (although I believe we are asked to provide slightly more data than the guidelines require). It does have some tracking and assessment tools which we use quite effectively, but they are not complete, nor are they flexible enough to allow us to tailor the results the way we do in our tracking spreadsheets.

Obviously it would be preferrable to keep it in the mis, but until the features are available, we have no choice...

[garrysaddington]

That document seems to highlight what you should be doing...

Not what you should be doing, what your MIS supplier should be doing, there is a separate document for schools. This is the other doc.

Management Information Systems (MIS) and pupil tracking: Users' guidance

[LosOjos]

I think with SIMS in particular, the tools are all there to get the info you need, but where it falls flat is the very limited functions available in AM7, making it much easier in most cases to extract the lot to Excel and mess with it there.

If Capita improved the functions available I think AM7 would be up to the task, but they'd need to at least make if statements have a LOT more cases, 3 and an 'else' doesn't cover it, plus nesting functions: it's simply not practical to have 8 columns hidden away just to get to one result, it makes marksheets huge, messy and difficult to manage.

It'd also be nice if you could add whichever columns you wished to a marksheet from elsewhere in the system, the available 'additional student info' columns are extremely limited when you look at the vast amounts of data that are stored in SIMS, which must all be linked up to a student by some common identifier, so why can't we get at them?

One thing that has always bugged me too is how awkward it is to switch between result sets, you have to either go in to each column individually and change it, or delete the lot and add them back in with the new result set: if you could highlight a range of them and update the result set in one go, that'd speed things up a hell of a lot.

[Tyiell]

I think with SIMS in particular, the tools are all there to get the info you need, but where it falls flat is the very limited functions available in AM7, making it much easier in most cases to extract the lot to Excel and mess with it there.

If Capita improved the functions available I think AM7 would be up to the task, but they'd need to at least make if statements have a LOT more cases, 3 and an 'else' doesn't cover it, plus nesting functions: it's simply not practical to have 8 columns hidden away just to get to one result, it makes marksheets huge, messy and difficult to manage.

It'd also be nice if you could add whichever columns you wished to a marksheet from elsewhere in the system, the available 'additional student info' columns are extremely limited when you look at the vast amounts of data that are stored in SIMS, which must all be linked up to a student by some common identifier, so why can't we get at them?

One thing that has always bugged me too is how awkward it is to switch between result sets, you have to either go in to each column individually and change it, or delete the lot and add them back in with the new result set: if you could highlight a range of them and update the result set in one go, that'd speed things up a hell of a lot.

I agree with all of that - and could add a litttle more: Performance Management is getting better for pupil tracking in AM7, but there are still issues with filter groups: you can filter by SEN Status, but not by SEN Code. You can filter by EAL, but not by ethnicity. Not sure why they don't just let you filter by anything recorded - that is what databases are designed to do after all!!!

[Matthews]

Hi Kiran

There's only so much that AM7 can do (As far as i know)
We do have things set up that tells us how many students getting 5 A*-C etc... but subject by subect they want to know percentages getting 5A*-C and considering we do 15 to 20 subject depending on the year it all becomes a little laborious to right click above each column (aspect) to get those percenatges. If it's a case that i'm doing it the longest way then i'd appreciate your advice on the quickest....

Currently i'm right clicking on the headings and asking for grade distribution to find the percentages. I'm then having to do group filtering to find the percentages for each category (Fems,Males,SEN.FS,EAL & Eth) so as you can imagine this becomes very long winded. Each subject each year for each category. Ofcourse we do have things in place that tell us 3 levels progress ect... but i feel as though i'm having to look in several different places to compile one analysis report.

Any help is appreicated Kiran

[Matthews]

Do you mean like your MIS supplier should have already been doing, without making schools do the work. See the link below (from 2008).

Management Information Systems (MIS) and pupil tracking: Developers' guidance

What is the point of collecting all the data in a MIS to then have to export it somewhere else to do something useful with it. Why not have a spreadsheet open in the staffroom to collect the data and cut out the MIS? Then ask for a reduction in annual fees for your MIS.

Hi garrysaddington

It doesn't half feel like you're completely on your own when it comes to manipulating SIMs to produce this information. I would say i'm quite good at using and manipulating SIMs or AM7, but this one has stumped me.

If i don't get this one sorted then i'm stuck doing a full two week painfully boring step by step day by day analysis, with mixture of extracting,exporting,manipulating and calculating tons of data.

[Matthews]

That document seems to highlight what you should be doing without really going into how - most particularly it is not specific to Sims. I can't speak for CMIS, but SIMS does not currently have an in-built solution to provide the extensive data analysis we are asked to provide (although I believe we are asked to provide slightly more data than the guidelines require). It does have some tracking and assessment tools which we use quite effectively, but they are not complete, nor are they flexible enough to allow us to tailor the results the way we do in our tracking spreadsheets.

Obviously it would be preferrable to keep it in the mis, but until the features are available, we have no choice...

I totally agree!

[kiran]

Hi Mathews

In part I agree - it isn't the most friendly; but you can use aspect analysis which does help the with GCSE WAGs and Target Grades with filters for SEN etc. Have a play - that's all I do and see if it works. Apart from that we do much analysis in Excel as the LA use it for their tracking too!

[willkeddie]

Tyiell - I am currently investigating ways of doing this. I would really appreciate a copy of that spreadsheet if at all possible. I could send some biccies also - but they may go a bit soft in the post!

[Dancer]

As a SIMS user I feel that SIMS actually does quite lot of analysis within both Assessment and Performance.
Aspect Analysis quickly calculates A*-C etc for every subject in any assessment
Resultset anlysis allows the tracking progess within a subject across many assessments
Group analysis allows access to many different groups
Chance analysis is great for monitoring levels progress, stalling students, comparing against targets etc.

My main complaint that it is essentially me who prints out a load of tables from performance and distrubute them to HOD/SLT etc after each assessment and that it would be very difficult to extend its usage to the average person who uses the data. This undermines the data mining capabilities. I'm starting to look at tracking grids which means that much more is setup before hand but are relatively easy for the relevent HOD/SLT to run.

And the wierd filters where some fields are available in some filters and not in others (I think already been mentioned in this thread). You can set up a group analysis and then filter the groups on SEN Status but not set a SEN Status as a group. If I remember correctly you have different fields if you set the group filter on a marksheet template and in an indivdual template.

[ChrisH]

I am like DHicks but my routines are in VBA and only I know how to use them as they are a little clunkey. I have programmed custom functions with usually take columns on another worksheet and one or two filters like FSM and Male or SEN and not gifted etc. I then populate the various tables with these functions. I have some nice short routines that will work out who has 5 or more A* to Cs and if that includes maths & english etc. The core is pretty flexible so if anyone has any particular requirement for a macro for excel then let me know and I will see what I can do.

[CAM]

We use a modified version of the KS5 ALPS system. It is exported from Assessment Manager to Excel, residuals are generated and then I use a custom-built VBA script to apply a 4 colour traffic light color scheme to the grades. At the side is the total residual per student and it's sorted from high to low. At the bottom is the %A*-C percentages and the difference in the total points values between the current set of collected grades and the targets for each subject.

[Setra]

1. What kind of assessment analysis do you presently do in your school?

For example do you produce: Student/Subject Residuals, % of students achieving certain grade ranges for the core subjects?

Chances, On Target, Residuals, Range achievements A*-A, 5+ A*-C with Ma & En, 5+ A* - C, Achieving 5+ A* - G, Achieving 1+ A* - C, Achieving 1+ A* - G
All with Gender splits, Class splits, SEN, FSM, Ethnic, Language and Looked After as well as whole subject and whole year all requiring different aspects.

2. What kind of trend/predicted analysis do you do?

Very little at present; conditionally formatted spreadsheets, mostly, little grapic analysis.

3. How do you make use of “Value added” data such as FFT, Yellis, MidYis, Raiseonline?

We only use FFT and we use it equally with Teacher Assigned targets.

4. How much of this analysis is done in your MIS?

The A-Level chances are. Mostly.
But they are then exported to Acrobat to be coloured in - they're obsessed with traffic lights here...

5. How much of the analysis done outside of your MIS – say in Excel?

90%

6. How to you choose to present your data: Tables? Bar graphs? Line graphs? Colour coded conditional formatting?

Tables with traffic light formatting.
Chance matrices with traffic light formatting.

7. How often is this analysis done?

It is done for every year for every report - so 7 times a year.
The there's the big Exam Results stuff in the Summer. I've not been there yet...

8. Who does this analysis in your school?

Me! Except 3 of the Maths teachers help do some of the summer stuff (adding in the previously attained AS and GCSE grades) because of the short timeframe.

9. What problems/difficulties do you encounter with your current analysis process?

Massively time-consuming, esp. if splitting by calss - we have maybe 120 classes in each year.
Summer Exams only accept that season's results, so GCSE and As levels taken in previous years are not included in SIMs-generated analysis - hence the Maths teachers.
I'm currently working on a MAHOOSIVE spreadsheet to accomodate those. Apparently.

10. What features and functions would you like to see in assessment analysis module?

Choose the results to include. Colour-coding. Quickly flick between class and whole year analysis.
Also, pre-set templates to prevent creating a billion - one for each subject each slightly different.

Hope it helps.
B