Ofqual and DfE studying ‘feasibility’ of ‘fully digital’ exams

[DrCheese]

https://schoolsweek.co.uk/ofqual-and-dfe-studying-feasibility-of-fully-digital-exams/

 

lol at this

 

The exams regulator is undertaking a feasibility study alongside government on “what it would take” to make GCSEs and A-levels exams “fully digital”.

 

hmm I'm pretty sure any of us could tell them "What it would take"

 

Some exam boards are already piloting on-screen assessment, but research by AQA last year found teachers’ biggest barrier to digital exams was a lack of infrastructure.

 

There we go

[3s-gtech]

A solution looking for a problem. Putting technology in the way of doing something is kind of the opposite of what we’re trying to achieve - the tech is here to make everyone’s lives easier, not massively more complex.

 

I’m not a luddite, but having actually sat on-screen exams when I were a lad, paper was better and still is.

[localzuk]

The main barriers to this would be lack of devices, lack of good wifi, and lack of charging capability.

 

Even if a school has 1:1 devices, these couldn't just switch between "day to day" and "exam conditions" use, so you end up needing a set of dedicated "exam computers".

[rogerdnixon]

So we are 1:1 with Chromebooks across most of our secondaries now and use Trelson extensively for exams and it works really well with minimal fuss. However, if you don't have the devices and the required robust network connectivity - thats going to kill it dead.

[2ilent8cho]

We are 1:1 , so long as it is web based or has an iPad App we are good, MDM set single app mode for the exam, devices will be locked to that app.

[dmj]

Even if a school has 1:1 devices, these couldn't just switch between "day to day" and "exam conditions" use, so you end up needing a set of dedicated "exam computers".

Not at all. We managed to use 1:1 computers in mock exams back in 2017 by just switching chromebooks between exam mode and user mode. IIRC we used this: https://www.digiexam.com/

We did it as a trial for year9 as they had all been using 1:1 since year 7 so would have qualified for 'normal way of working'. I left the school after that so i'm not sure if they persued it.

 

Exam boards would save millions if this were rolled out, but I somehow doubt their shareholders would let them pass it onto schools to pay for the 1:1 schemes.

[Jamman960]

This could work out very well for schools and pupils if the exam regulators are able to state that exams will be digital from ... - funding would need to appear, ensuring pupils are digitally competent ready for their exams would be a great motivator.

 

They really need to look at the quality and robustness of the software platform being used, we've had no end of trouble with the GL Assessments platform failing to submit results at the end of tests - this cannot happen for some poor kids GCSE, IT managers really need to have confidence in the solution otherwise it'll be our necks on the line when pupils start failing exams because of IT issues.

 

If these tests could become adaptive they'd be able to assess pupils levels within fewer questions & time which could also help reduce the burden of marking, requiring pupils to be digitally literate to be able to access the exams in a fair way across the board would be a huge step forward and would go a long way towards ensuring they are ready for university and ultimately the rest of their lives.

Edited June 30, 2023 by Jamman960

[Linfit]

The way to do this is to implement on demand testing and move away from the everybody sits the same test at the same time model.

 

This would reduce the burden on schools as we wouldn't need 200 laptops/devices all in use at the same time.

 

The issue wouldn't I think be a at the schools end anyway, it would be the complete inability of the Exam Boards to run IT systems (I work closely with our EO and this year has been terrible from all three boards), the massive bank of questions that would be required to avoid the exams being too predictable, and the quality of the software platforms they use.

 

At the moment, you would never get them all to agree to use the same platform.

 

Yet another reason why the current boards should be abolished. I have thought for some time there should be just one board for England - works well enough in Wales Scotland and NI.

[Andrew_C]

And what, pray, happens when your machine restarts in the middle of what you're doing? As my expensive Surface has just done?

Lost time? Check.

Lost work? Check

Stress? Well, not this morning as I've learned not to care too much, but I'm not 1/2 way through writing an English essay that's just gone in a puff of electrons.

[localzuk]

The "everyone at the same time" model is interesting, as it would have a rather large backend requirement as well. 620k children sit GCSEs each year, for example. And they all happen in a small number of weeks. The servers behind those platforms would need to be incredibly robust!

[Brax]

The "everyone at the same time" model is interesting, as it would have a rather large backend requirement as well. 620k children sit GCSEs each year, for example. And they all happen in a small number of weeks. The servers behind those platforms would need to be incredibly robust!

Very good point, when the whole Surpass SecureClient software was rolled out and one of the first exams took place their servers couldn't handle all the students taking the exam in the country at the same time. It took pretty much a day before it was able to be submitted. And that was just A Level students not even GCSE's! Edited June 30, 2023 by Brax

[Ketlane]

Looking at it from the candidates' point of view, so many of them now do almost all their writing electronically, so sitting an exam of 2 hours or so with pen and paper is a very unusual experience. We can see the quality of writing decline as you go through the question paper from Q1 to the end at Q10 or wherever.

From the examiner's point of view you would avoid difficulties of reading handwriting - some has to be seen to be believed. Some marking could be automated - at a trivial level examiners wouldn't have to check blank answers, but also one-word answers or simple maths answers could be automated easily, leaving examiners to do the difficult (and more interesting) bit.

I agree with the general view that the devices have to be in plentiful supply, utterly reliable; that the underlying software has to be bomb-proof; and that the connectivity and server end has to be failsafe too.

Electronic assessment allows all sorts of ways to make exams better at measuring candidates' knowledge and skills - just making what we do now work on-screen is only a start. IB has been looking at it - see https://www.ibo.org/programmes/middle-years-programme/assessment-and-exams/onscreen-examinations/

Declaration of interest here - I'm marking on a GCSE which has over 800,000 scanned responses to mark. That's just one paper. Multiply up by the different subjects, the multiple papers, the different boards (someone will tell me off - they are Awarding Bodies and don't like being called boards), the different exams (GCSE, A Level, BTEC,....) and it's a mountain of data.

[FN-GM]

Even if a school has 1:1 devices, these couldn't just switch between "day to day" and "exam conditions" use, so you end up needing a set of dedicated "exam computers".

 

It requires a whole new way of thinking.

 

Exams in Australia are taken on personal BYO 1:1 devices. But the conditions are the same. They aren’t closed book exams like most of the ones in the UK.

[dmj]

Very good point, when the whole Surpass SecureClient software was rolled out and one of the first exams took place their servers couldn't handle all the students taking the exam in the country at the same time. It took pretty much a day before it was able to be submitted. And that was just A Level students not even GCSE's!

 

Worrying, scaling out servers on demand is about the easiest bit to this. Then again, it is exam boards we are talking about.

[Brax]

Worrying, scaling out servers on demand is about the easiest bit to this. Then again, it is exam boards we are talking about.

It is, to their credit though they have improved the software and there have been no submission issues this year but it was painfully obvious when it first came out that it wasn't 100% ready.

[CHiLL]

Where are they all going to sit the exam? In the hall/gym like existing? Good luck getting good enough power and WiFi for that! Or in the existing IT suites (assuming you have enough)? Then do you need a lot more exam invigilators and other necessary exam resources per room?

[dmj]

Where are they all going to sit the exam? In the hall/gym like existing? Good luck getting good enough power and WiFi for that! Or in the existing IT suites (assuming you have enough)? Then do you need a lot more exam invigilators and other necessary exam resources per room?

 

When we did out trial we had only half a year group, which was 150 students at a time. Wifi really wasn't an issue with 4AP's and at the time we did some calculations that it would easily scale to a full sized group. Of course it will depend on current setups - but this was six years ago so and my school wasn't that far ahead of the curve.

 

We had a handful of spare chromebooks to hand out. We only ran a few sessions but they all went quite smoothly. I suspect it would be much more difficult if it were the exams team and not the IT team running those exams though.

 

It's all workable, but can't seem to help thinking that this will end up be a way of letting the exam boards make more money at the expense of schools.

[pete]

A budget lever to push out 1:1, funded by the DfE with guaranteed refresh money every year? Sure. It would also let me set MFA on for all students relatively easily (device + creds).

 

Do I think the exam boards would properly provision their backend infrastructure? Not until the first round of firings after it goes horribly wrong in year 01.

 

--

 

Properly done (MidYIS, ALIS, LiBF) online testing and exams work fine. I just don't trust AQA, Edexcel or OCR to be able to deliver them. WJEC are a bit better overall (compared to the other 3) but their "upload your film studies moderation examples" portal is awful.

Edited June 30, 2023 by pete

[sigma]

There is a consistent direction of travel here. Take maths. Do schools need maths teachers or can anyone supervise pupils while they sit in front of a device on Sparks Maths or whatever the current fad is? How long before the current fad gets accredited with being able to award a GCSE to pupils passing the game/quiz or whatever?

 

/rant

[Ketlane]

One of the good things I've seen is responsive exams - as the candidate answers, the software assesses them and - it they are doing well - increases the difficulty of the next question; or reduces the challenge. The candidates emerge having answered different questions, possibly in a different order, but all of them have been challenged. They all say that it was difficult but they thought they'd done well. No need to romp through loads of dead easy questions before you get to the meat of the exam; or no need to stare in despair at the harder questions you haven't a hope of answering. This is a much improved version of Foundation and Higher papers!

[smurfomatic]

Responsive exams are great (15 years of invigilating online NGRT along with other tests) but it removes the option to go back and check/change your previous answers, which may not be too much of an issue with GCSE and A-Level, but when a primary age child accidentally skips three questions in a SAT and can't do anything about it...

[Ketlane]

Responsive exams are great (15 years of invigilating online NGRT along with other tests) but it removes the option to go back and check/change your previous answers, which may not be too much of an issue with GCSE and A-Level, but when a primary age child accidentally skips three questions in a SAT and can't do anything about it...

My experience is with Getintu which won't let you skip questions. This is a whole different way of examining as no-one (well, certainly not everyone) does the same questions.

[dmj]

One of the good things I've seen is responsive exams - as the candidate answers, the software assesses them and - it they are doing well - increases the difficulty of the next question; or reduces the challenge. The candidates emerge having answered different questions, possibly in a different order, but all of them have been challenged. They all say that it was difficult but they thought they'd done well. No need to romp through loads of dead easy questions before you get to the meat of the exam; or no need to stare in despair at the harder questions you haven't a hope of answering. This is a much improved version of Foundation and Higher papers!

 

Sounds good, but in practice it would take a lot of client>server calls to achieve or a very heavy client side application. Either way it's going to be a way off in the future considering the limitations others have discussed with their current network setups.

[smurfomatic]

My experience is with Getintu which won't let you skip questions. This is a whole different way of examining as no-one (well, certainly not everyone) does the same questions.

 

This was with GL Assessments, which hadn't changed in my time working with them. There are only about ten different comprehension passages, so knew most of them off by heart!

[free780]

I think the only way it would work would be.

Exam board brings devices and servers in a closed wired network. No dependency on the BAU network devices. The equipment would have to be placed in exam halls for the duration of the exam season. Any reliance on cloud is bound to encounter issues. Using separate equipment would also ensure any advantage/cheating would be very difficult.