[sims] Progress 8 Measure

[tombry]

So I'm sure most of you have seen the document from the dfe laying out their current ideas on the progress-8 measure, with promises that we'll have the full one at the end of February.

 

I'm starting to build out some marksheets to look at my current y9s and see how they line up under the progress 8, and was doing well until I got to the actual progress 8 measure.

 

I've got my Y9s in with their estimated grades, going across 15 subjects. Going from the dfe guidance I need to count their best 8 grades with the following rules:

 

2 of the 8 have to be English and Maths

3 of the 8 have to be from Ebacc subjects; Science, ICT, Geography, History, Spanish, French (easy enough, Max 1, 2, 3 columns pull that)

the last 3 have to be from the Ebacc, as well as Art, Drama, Muisc, Computing, and any Vocational courses that make the list.

 

and here is where I'm having a problem, as if I use Max 1, 2, 3 columns, I'm going to get overlap from the Ebacc subjects (I'm sure) - so how can I pull the highest 3 grades that aren't included in the Ebacc Max columns?

[Dave_Bensley]

my understanding is that you rank the ebacc subjects all the way from 1 to 6 and then in the final grouping only include the ebacc subjects ranked 4 to 6, avoiding any overlap?

 

have not go round to building it yet bethink this should work - let me know f I have helped or confused please.

 

dave

[Seb1780]

.....where I'm having a problem, as if I use Max 1, 2, 3 columns, I'm going to get overlap from the Ebacc subjects (I'm sure) - so how can I pull the highest 3 grades that aren't included in the Ebacc Max columns?

 

Would something like this work?

 

max1,2,3(min1,2,3(ebacc),other subject1, other subject2,.........))

[PhilNeal]

Progress 8 isn't completely defined as yet. You can get close to what has been put up within AM but some of the bits around the edge need to be hard coded unfortunately. The plan is to get a report into AM probably with the autumn release. Schools are not required to report on Progress 8 until next year.

[tombry]

@Seb1780, you put me onto the right way of doing it - I just need to include the Max 1, 2, and 3 columns in my 4, 5, and 6 checked columns.

@PhilNeal,

 

I appreciate that we won't have to report on it until next year, but I have Y9s that are choosing their options, and will start those options in May/June when the Y11s leave for study break, and would like to see what they look like. I also know that it's not been completely defined yet, but it has basically been frameworked, so it's easy enough to get a start on it.

 

As for the edge bits that would need hardcoding, I'm not sure what those would be - it seems to be a very simple measurement, and outside of the issue I was having above, I think I'll be able to get it set up today. Can you explain what you mean?

[PhilNeal]

@tombry I've spoken to the experts and you are right. It can be done in marksheets but its quite convoluted and a hard coded report will be produced.

[tombry]

@PhilNeal

 

Either I've way oversimplified this, or I'm confused by what your experts consider to be a convoluted mess!

 

I've got Max 1-6 columns set up pulling in my top 6 Ebacc scores, I then have other 1-3 Max columns pulling from EBacc Max 4-6, as well as my other subjects.

 

I've doubled English, throw it all together and get my scores.

 

Seems far less convoluted than the 60 step process I was sent on how to set up tracking the EBacc as the nested if statements treat a blank space as greater than a number (which still isn't 100% accurate, but by now I know who my 4 or so false positives are)

[davidpott]

@tomby

 

I think you've done it! I've just had a play with MAX formulas as you suggest and (with a minor alteration) - it works.

 

I've included the updated advice about doubling the highest score of Eng Lan or Eng Lit, then using the second higher of the two with the Ebacc 'pool' (see https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/269438/update_progress_8_measure_secondary_school_accountability_framework_reforms.pdf)

 

 

The first column is just the math grade - 'as is'

 

Next, a couple of MAX formulae identify the highest and second highest eng lang or lit grade, then doubles the highest.

 

A second set of MAX Ebacc 1 to MAX Ebacc 6 formulas look across the remaining EBacc subjects (but not english or math)

 

A third set of MAX Other 1 to MAX Other 3 formulas look across the second highest english column, the MAX 4, 5 and 6 Ebacc subjects and all the other non EBacc subjects

 

The Progress 6 Column uses a SUM formula to add together MAX EBacc 1 + 2 + 3 and MAX Other 1 + 2+ 3

 

Finally the Progress 8 column uses a SUM formula to add together Progress 6 + DOUBLE English + Maths Grade

 

I need to go and lie down. What do you think?

 

Forgot to double weight maths - but you get the idea.

Edited February 25, 2014 by davidpott

[tombry]

@davidpott

 

Exactly what I did, although as I only have y9s not separated to eng and eng+lit yet, didn't get that set up, so props to you on that, I'll make sure to take that in when we split them off in a couple months.

 

Edited to add:

 

Set yourself up a new gradeset using the p8 points, aspects to hold them, and a mark sheet to copypasta the grades from one to another, take about an hour or so, but you'll need it soon enough

Edited February 25, 2014 by tombry

[davidpott]

Good advice about the new 1-8 values for the grades A*-G - I've not really taken the implications on board yet...

[Ecclesbury]

These marksheet templates look good.

 

One thing to bear in mind is the English is only double weighted if both English & English Literature have been studied (with, as you say, the higher grade counting.) Easily achieved in a template, but you may need to tweak what you describe above (off the top of my head I'm thinking an extra GRADE TALLY COLUMNS counting the 2 English columns and then multiply the English MAX points by this value.)

Edited February 26, 2014 by Ecclesbury

[tombry]

well for the first year it won't matter - as the English Exam they take carries some Literature study in it - at least that's what the fact sheet from the gov says, and I'm just trying to give our Head something to look at to prepare him for how different it looks compared to our current methods.

[Ecclesbury]

Ah yes, that's true. A lot of this will depend on each schools particular curriculum offer as doing a Double English course, or one with some literature, as you say, will still count double.

 

Of course they could change all manner of other things anway, but I'd hope that large scale changes won't be

[Banjo]

If you have access to FFT data you could add in est. 'A' for those subjects, and subtract the predicted grades from them, and work out whether you will be below the Dfe floor (avg -1/2 a grade difference), or averaging +1 grade above and get a free pass from an Ofsted inspection for 12 months.

[tombry]

well I know what I'm putting together tomorrow

[NotSoFast]

I've been playing with similar ideas. Just be aware that this will be a rough approximation. We don't yet know how the school's target will be calculated and doing it this way doesn't account for other schools offering students a different curriculum (for one thing).

[tombry]

Really rough for sure, but will at least give me something to go to with for showing the head how it's changing.

[njw499]

This sounds fantastic, is there any chance you can share the marksheet so I can have a look at

[tombry]

I don't have them built as aspects yet, just formula reviews - needed to proof of concept it before taking it to the full on ends.

 

is there a way to export it nicely?

[Banjo]

I've been playing with similar ideas. Just be aware that this will be a rough approximation. We don't yet know how the school's target will be calculated and doing it this way doesn't account for other schools offering students a different curriculum (for one thing).

 

FFT A methodoloigy and how the DfE will calculate the subject level estimates should be fairly similar. The errors will creep in when you look at your own schools predictions of what students will achieve.

[tombry]

I put A, D, and our internal targets in, as well as our estimates.

 

I think in the first instance it's going to make the head question how some students have the targets they do - when they are 5 grades over internal targets, but 5 grades below FFTA targets, there is probably something wrong with their internal targets.

 

I'm just happy I have it set up now really - I'll worry about finalizing it after the full spec comes out, and I'm guessing after the releasing potential event I attend next week.

[vikpaw]

I don't have them built as aspects yet, just formula reviews - needed to proof of concept it before taking it to the full on ends.

 

is there a way to export it nicely?

 

Routines | Data Out | Assessment | Export

 

Haven't done it for a while, but i think you can dump a load together as an XML for someone else to view. You'd also need to do the aspects that are contained in it for it to work. The recipient needs to import them.

 

The system i inherited was built like this, except the mods made on top didn't follow the same pattern, and aspects that should have been grades were added as integer and the whole thing became a mess, but it can be v.useful.

 

We should do this more often here, if it works, so others can share the cool templates and the hardship of the formulae creating, but when everyone has a different set of aspects to begin with it becomes a problem.

[tombry]

as @vikpaw has told me how, I'll export them out later, or tomorrow even.

 

I'll give you the aspects and the template - should be enough, as the template only has the aspects and the formula review columns.

 

bonus for me, y'all can give me feedback on tweaks.

 

keep in mind, I don't have to figure out the English on this, as it's year 9, and they haven't been slotted yet.

 

sometime in June I'll be doing that one up.

[Ecclesbury]

I may be wrong, but I'm pretty sure that if you export the template on it's own, when another user imports this template it will create any associated aspects, grade sets and result sets that don't already exist in their database. In other words you don't have to also export the aspects and such, the template export will do all that for you. However, it does mean that a user importing your template will end up with a load of extra aspects in their database. Not only this, those imported aspects will not be editable so you can't be sneaky and just rename all the aspects that you have created.

 

Final thing, even if you had aspects with identical names, when importing they will not match up. SIMS will do the usual thing of creating a new aspect called ASPECT NAME~1 and GRADE SET~1 etc. This is because, like timetable elements, all assessment manager elements have associated GUIDs.

 

To cut a long story short, if people plan to take someone else's template and use it in their own database it is going to be much easier to import the template to have a look at how it all works, then delete it and re-create your own version.

Edited March 4, 2014 by Ecclesbury

[Ecclesbury]

That's not to say you couldn't open the raw XML and tinker with the GUIDs of course! But I believe the imported template itself would still not be editable.