Head of of Ofqaul asks: why aren't students using computers in exams?

BBC Radio4 Today Programme 25/02/2011 0732
Should computers replace pen and paper in all exams? Isabel Nisbet, outgoing chief the Ofqual qualifications watchdog, argues that GCSEs and A-levels will become "invalid" for digitally native pupils if writing materials are retained. She debates her proposal with Dr Sheila Lawlor, director of centre-right think-tank Politeia.

BBC News - Today - Today: Friday 25th February

I'm quite excited to hear this debate out in the open. Having used a PC in my own GCSE's many many years ago, and run GCSE's in a SEN school with 60+students all sitting their public exams on laptops, 9 years ago I am continually disappointed by the lack of progress in this arena.

In my experience it is easy and cheap, assuming your organisation has invested in appropriate technology already. However 60 is not the 300 that I'd need to do the same thing in a large secondary today.

The security fears she raises are easily addressed. In fact that this link here is Using Windows 7 and UMRA for Controlled Assessment almost exactly as I did it 9 years ago.

The question is how do we get enough devices to pull it off? Or is it better to use the controlled assessment model and abandon the idea of terminal examinations simply because they cannot be adequately assessed in todays' environment?

Just caught the end of this as Ms. Nisbet said she was going to Singapore and a little bit about Chinese calligraphy. I'll try and catch up with this when I take a break from decorating. I've a soft spot for Ms. Lawlor, it's the Great Grimpen Mire.

Quote:

---

Originally Posted by psydii

In my experience it is easy and cheap, assuming your organisation has invested in appropriate technology already. However 60 is not the 300 that I'd need to do the same thing in a large secondary today.

---

It's logistics. All of our (~880) kids do exams in the summer and GCSE and AS/A2 mocks are done in February. Assume each A-level kid (~280) is doing at 4 exams of at least an hour each and GCSE (~600) are doing 9-10 exams of at least an hour each. This discounts multi-day practicals (Art, Food tech, Chemistry etc).

We have 3 halls, which can seat 200, 250, 100. So 550 kids can be doing exams at once and we need 3-6 invigilators. We'll also have smaller exams going on in other rooms.

Then consider our IT setup is based around the 30-34 kids in a classroom unit. Even with all the computer rooms full, there's no practical way we can run kids through exams at the same speed. Plus trying to find reliable invigilators who can say "yeah, you've got caps-lock on" without calling IT support is trickier (in this neck of the woods) than you might think.

For 4 weeks in a year, it's crazy exam season. The rest of the year, there's no call to have hundreds of kids using computers in such a synchronised activity. This is before we start to consider that some lessons are being taught during exams and hijacking resources from them would be rather unpopular.

The logistics solution? Each kid has an exam-suitable device that they use as their primary computer in school. During exam season these devices boot into $other_os or (insert method of ensuring no access to local devices/data/internet) and load the exams from an in-house server via (decent, properly scaled, managed) wireless. Since each kid has one stashed in their schoolbag, it doesn't matter where they're taking the exam.

Xen client could do this if you had enough RAM

Quote:

---

Originally Posted by irsprint84

Xen client could do this if you had enough RAM

---

You'd still need physical hardware for each student though.....

One of the things that Isabel Nisbet said in the interview was along the lines of "this has always been considered too difficult - I'm now just taking it off the 'too difficult' pile" to get people thinking about it" and I think that's fair.

There are loads of reasons why this could be a good idea and loads of potential problems. We're using it for a limited number of exams (mainly where you're being tested in things like programming or where you need to use a particular piece of software as a rather special calculator)

I'd guess that in schools it would make sense to start by using computers for exams where they provide a real benefit in the same way that calculators started being used in exams because it meant that you didn't have to make the arithmetic easy and could concentrate on the physics, chemistry etc.

It isn't going to happen this year but I'm sure it can happen. Many schools have a good framework in place (eg the facility to lock down machines, reset machines to a known state at the end of a day, restrict internet access by user group or machine group) and so on but getting enough hardware will be a challenge. Is this the time when we change away from everyone doing the same exams at exactly the same time?

Quote:

---

Originally Posted by srochford

...but getting enough hardware will be a challenge. Is this the time when we change away from everyone doing the same exams at exactly the same time?

---

I can't see it being done any other way, a typical class of 30 PCs has the PCs too close together to use all 30 under exam conditions, unless you were to put a partition between them....

What if the exams were based on a bank of 1000+ questions? then there would be less of a need for the entire country to sit the exams at exactly the same time and would allow for smaller groups to sit the exams at different times

I can't see it working at all without changes to the exam format.

Quote:

---

Originally Posted by Jamman960

What if the exams were based on a bank of 1000+ questions? then there would be less of a need for the entire country to sit the exams at exactly the same time and would allow for smaller groups to sit the exams at different times

I can't see it working at all without changes to the exam format.

---

I started typing something along those lines and then hit some random key combination that sent IE back a page.....

My thought on that was that some exams build on the previous question, or have a flow to them so it wouldn't work in all exams, but for those exams that could be done this way it would allow more PCs to be used in a room.

But as you say, there needs to be changes to the exam format, it isn't just down to a technological response.

annoyed about this. IMHO it is mostly about exam boards wanting to pass their costs of marking etc onto the school. ie - the schools pay for equipment based on the exam boards criterion which is contained in the "Instructions for the conduct of the examination" (ICE) document. The ICE document rarely pays attention to the pedagogic setup of the IT suite, and focuses entirely on the exam board saving money. With the already high costs of the outsourced exam boards, they should stump up the costs of their examinations!

I was talking about this at bett, I think it needs to be effectively a different test for each student and also they need to be allowed to take the exam when they are ready to take it.

Ben

The "different test for each student" is already there in some of the tests - the Promissor stuff was doing this years ago so the exam seating model becomes much less important.

There are some exam boards which allow exams to be taken on demand - not sure if this is true for any UK qualifications but it certainly happens for some of the Cambridge International stuff (I used to set the papers!). The issue with this in many schools is that the exams end the year and nothing happens after the student has done the exam but you could have a different system.

It's not just about passing costs on to schools (although I'm sure there's an element of that in there) but making exams fit in better with the teaching and learning. It's not going to happen tomorrow but we can start thinking about how we would do it and how we make sure that the system develops in a way which doesn't cause unnecessary expense, disruption etc etc.

I think if ofqual really want to go down this route then they should standardise an online testing process or online exam centre. many exam boards with their own software just makes a mess. (sometimes incompatible with each other, IIRC there was an exam board a couple of years back that required IE6 for one piece of coursework and IE7 for another..)

This really isn't that new in their discussions though ... http://www.edugeek.net/forums/genera...ur-behalf.html saw me and Chris go to talk with many others about it and the biggest barrier is how the curriculum is set up in the first place of only testing at the end of defined periods, not when students are ready.

We are back to discussing the KS3 ICT tests again ...

The key part of this is the comment from the OCR

Quote:

---

However, a spokesman for OCR said the board's focus "was not to make existing paper-and-pen tests electronic but to explore ways that computers can add real value to assessment".

---

BBC News - Ofqual head: End paper exams for digital generation

And this is where the nail is hit firmly on the head. The real question should be "why are students still doing (so many) exams", not how can we transfer exams to a computer.

The main reason for persisting with exams is a paranoia about plagiarism - and an invalid belief that exams are the only form of assessment which can't be plagiarised - and a sense of that's how it's always been done so it must be right.

But exams are flawed because they take place in an artificial setting, take no account of differing learning styles. It's very much open to debate whether exams actually assess anything useful.