BBC News: 'Online tests to replace paper exams within a decade'

[Andrew_C]

Look at the Driving Agency tests as an example. 45 mins per test - each session is staggered so that the candidates are seated within 5-10 mins, tests are started and then end, and 5 mins to clear them out. Next session can then begin.

But that's a simple multi guess, presumably with each candidate getting a perm x from y set of questions. Nothing like 150 kids sitting down to write an essay on Macbeth.

 

Oh wait, in another thread we are going to scrap that...

[localzuk]

If you do it as above though, you don't need a massive new infrastructure as you can then fit it into your existing labs without too much disruption (and let's face it, there's always some disruption around exams time so that's nothing new). Also, I'm not sure just how much extra work there would be for the exam boards as essentially it's just an extension of existing practices. Add to that the exam boards' savings (both in money and effort) in printing and delivering all those papers - plus the schools' savings in buying and storing them - and I'd imagine it would at least balance out.

 

Existing labs/suites couldn't be used though, as most schools don't have the rooms laid out in an exam layout. Not enough space between machines, able to see other screens etc...

 

Best I've seen in similar situations is only half the machines used, so as to space people properly.

 

Plus, that only works if schools still have enough suites. There are schools who have reduced the number of suites after introducing ipads for all pupils. The examination software I've seen all required Flash so you couldn't use the ipads either.

 

The cost to schools if this sort of thing happens will be pretty high, and will require a rethink of how exams are taken on a more fundamental level than paper or computer.

[rad]

I am currently doing our CATs test on the testingforschools website. One room, one class 3 periods of lessons. 6 days to do the entire year 7. It takes me 20 minutes logging them on and launching the website. If we had to do the setup for an exam, 12 classrooms for the non statemented students, another 3 rooms for the Statemented students, that's just for the 300 year 11 students.

 

What about the scheduled lessons that take place....sorry ICT teachers, you can't teach your kids, we need your rooms for exams. Like that's going to go down well.

[mrbios]

What happens when a machine crashes mid exam? What happens if there was a power cut? What happens if there's a bug in the software? The list goes on....

 

Seems like a stupid idea and is best left to pen + paper. I hate the amount of paper schools use, and think a hell of a lot more could be done to reduce their paper consumption, but exams aren't one of them.

[Ketlane]

Technology - or at least exam setting - will move on too. We already have on-demand papers; there are also computer-controlled adaptive tests that sense how well a candidate is doing and then choose which question to give next, easier or harder to suit. These adaptive tests mean that candidates doing the same exam at the same time aren't doing the same questions, means cheating is harder. Also means that top candidates don't have to do loads of stupidly easy questions, and that weak candidates don't get given questions they haven't a hope of answering. I think that ISEB is on to this one?

With typed answers, at least some marking could be automated - although human judgement will be essential for a long time to come.

And as we increasingly move to a screen-based communication system, the business of a pile of ruled A4, a biro and just under a square metre of plywood will be totally inappropriate for examining. There are tech problems, true, but the exam boards seem to have sorted out their end of things pretty well with just about everything electronic. And Moore's Law will help - the price and power of devices will continue to change very rapidly.

[Ephelyon]

Not enough space between machines, able to see other screens etc...

 

What happens when a machine crashes mid exam? What happens if there was a power cut? What happens if there's a bug in the software?

 

Obviously these are valid problems and, if the Government and exam boards do decide they're going down that route, come what may (cannons to the right of them, cannons to the left of them), then as the issue gets discussed more widely the solutions will have to be found. If we in educational IT are wise, we'll get in on those discussions in good time to ensure the best result and minimise the amount of crud falling on our plates that we hadn't been aware of.

 

Ironically, those IT suites designed with six or so computers on a hexagonal-shaped desk, which have been criticised so heavily from a T&L standpoint though many schools still have them, could actually prove ideal for an exam setting in terms of sneaking a sly glance at someone else's screen. Adaptive testing will also help there.

 

The remaining questions seem to focus on "What do we do when something unpredictable happens?", so if there is a technical problem or if the school loses power. As for bugs in the testing software, the responsibility for that lies with exam boards and at some point there has to be some trust in their internal QC procedures; if they fail, it's not on our heads. On the subject of technical problems with a particular workstation, that could be remedied through an online testing package that uses technology similar to Google Docs to auto-save almost constantly. The local IT team would need to have "hot spare" devices to swap in within 5-10 minutes, which is a manageable amount of extra time to allocate to a pupil at the end of an exam. They could then sign back in on the spare device and continue their work. The matter of "hot spare" devices for the IT environment in general is a strategic issue to discuss with SLT because it involves both resource management and money spent.

 

A power cut is a highly disruptive event that I don't have an answer for at the moment, barring even more versions of an exam to cope with it. Naturally there's also the issue of a sudden outage at the ISP's end, such that all the fancy cloud stuff I've just mentioned is rendered useless. That leads me to conclude that, because we can never rule out IT issues, the point of view from which change must flow needs to be how we address - nationally - the issue of pupils needing to resit all or part of an exam because of unavoidable circumstances. I expect that's not something we can answer here and now though.

Edited September 20, 2013 by Ephelyon

[Steven_Cleaver]

Interesting discussion.

 

How about this !!!

 

During the Exam period close the school to all years except those taking Exams so they have this as part of their Summer holiday so 4 weeks allow some staff off for two weeks, some staff for Invigilating exams any spare staff could concentrate on creating resources for the new year (which they should be doing as part of their holiday and some do well but others not at all) and have these signed off by their Line managers. Second 2 weeks have the other Staff on holiday and swap round. This would allow all IT suites available for exams and spare rooms if required to hold students with interactive boards so could watch tv do extra revision, between exams etc Careers advice etc. As would have plenty of facilities.

 

As someone has said extra kit in case the odd device crashes.

 

Also have one school in a borough where no exams are sat but students from this school go to different schools to sit the exams either themselves or bused and have a set of coaches on standby in case a schools systems did crash then Coach or bus all those students under supervision to the empty school. Obviously if more than one school crashes on the same day you have issues but can only cater for so many scenarios.

[rad]

Seriously though.... Kids will soon not know how to use a pen, hands up if you'll admit as an IT person to have cr@p handwriting cos you don't write much... I am one of those.

 

I think exams on ICT equipment is a bad idea, it's a headache sorting the SEN concessions out, making sure x computers are set up, let alone xxx

[Ephelyon]

It would be a bit difficult to close the school to everyone except Years 11/13, considering Years 10/12 are at a critical coursework stage as well.

[Dos_Box]

Several years back, Tony (Grumbledook) and I were asked to attended a meeting organised by Ofqual (IIRC) discussing the viability of this very subject. Whilst the exam providers and education sectors were quite enthusiastic both Tony and I threw some spanners in the works. Mostly these have already been mentioned and, from what I can remember we came up with the following points:

 

1. You would have to provide 1.1 computers/laptops in a space that is suitable for taking exams

2. You would then have to provide enough power and (importantly) secure networking, to the computers

3. If wireless networking is provided, how do you ensue the signal is not available for scanning outside of the exam environment?

4. The exams would be hosted online and steps would be taken to ensure that in the event of a crash they could be rejoined, but how do you calculate the additional time required should a machine bomb-out?

6. Who would hold, maintain and setup machines which have been setup in such a way to provide a secure and 'clean' environment for exams to take place

 

 

There were quite a few other points as well, but at the time we mentioned that we didn't think that the technology was quite up-to speed to run online exams in the 'traditional' manner exams had been held in the past. One solution that was suggested was that pupils would take an exam online when it was deemed that they were capable of doing so and in the event of a pass would then be placed on a higher level course.

It was all very interesting stuff, but the holes we found in the process at the time meant that we didn't think we'd be seeing it any time soon.

[CyberNerd]

I think that within a decade this is feasible because the price of a device that could complete such a task is (and always has been) dropping to the point that they are cheaper than textbooks.

There is a cultural difference in the UK that means schools usually provide the educational resources (computers) that doesn't exist in other parts of the world.

 

The main barrier is that the exams companies are not able to write software, and that schools are unwilling to provide 1:1 computing, The latter will change.

10 yrs back I completed an online/cloud based Cisco exam that would run securely on a multitude of operating systems/browsers. 10 yrs later, the exam boards are so closely tied to MS products that require an install in the traditional school owned desktop PC environment, this is always going to be an expensive option. If the government required the exam boards to up their game and write a platform that would allow true could based online exams(unlikely) then there would be a way forward. Until then it's blue sky thinking.

[Dos_Box]

I think that within a decade this is feasible because the price of a device that could complete such a task is (and always has been) dropping to the point that they are cheaper than textbooks.

There is a cultural difference in the UK that means schools usually provide the educational resources (computers) that doesn't exist in other parts of the world.

 

The main barrier is that the exams companies are not able to write software, and that schools are unwilling to provide 1:1 computing, The latter will change.

10 yrs back I completed an online/cloud based Cisco exam that would run securely on a multitude of operating systems/browsers. 10 yrs later, the exam boards are so closely tied to MS products that require an install in the traditional school owned desktop PC environment, this is always going to be an expensive option. If the government required the exam boards to up their game and write a platform that would allow true could based online exams(unlikely) then there would be a way forward. Until then it's blue sky thinking.

 

At the meeting all of the top exam companies were there and all of them either had, or were working on online examinations.

They are actually very much in favour of the idea as it saves them a fortunes in printing, distribution, marking, special needs provision and a whole raft of other small details. In short, they get to make more money.

Setting up an online exam infrastructure also has the advantage that many exams can be marked and the results returned in a very shrot space of time.

[CyberNerd]

At the meeting all of the top exam companies were there and all of them either had, or were working on online examinations.

They are actually very much in favour of the idea as it saves them a fortunes in printing, distribution, marking, special needs provision and a whole raft of other small details. In short, they get to make more money.

Setting up an online exam infrastructure also has the advantage that many exams can be marked and the results returned in a very shrot space of time.

 

The problem seems to be that their idea of 'online' is a local install with specific requirements that preclude BYOD.

[Dos_Box]

One major sticking point is that exams have to be undertaken in a 'secure' environment. Could a BYOD machine ever be classed as secure?

Tony and I had put forwards that any machines used for online exams should be owned/operated by the school/authority (perhaps even the exam provider). Imagine a pupils having a screen sharing app running on a BYOD device!

Couple this with the current thinking that all exams must be run at the same time throughout the country at the moment as well. How do you suddenly magic up tens of thousands of exam ready computers for multiple exams?

I'll have to admit that both Tony and I went away from the meeting scratching our heads at the complexity of some of the issues.

[CyberNerd]

I think that if this were to go ahead it would have to involve BYOD and the government would need to move away from the idea of sitting all students at once. I dont think either are likely.

Surly policing exams for BYOD is the same as ensuing students don't bring in their own written notes?

[Arthur]

Could a BYOD machine ever be classed as secure?

I imagine within the next 10 years we'll see more devices that include something similar to VMware's Horizon Workspace - a secure version of the OS that the school (or employer) can control without affecting the non-secure OS that the device owner uses for thier personal stuff.

[Ephelyon]

I think the difference with written notes is that getting them out, passing them around etc constitutes a physical action, visible from afar, that doesn't mimic legitimate activity, whereas screen-sharing on a laptop/tablet would look much the same as normal exam work from a distance unless you're paying very close attention to at least one of the screens if not both (difficult if they're on opposite sides of the hall), AND you know what you're looking for. Technical training for invigilators would be required there, and as always the cat-and-mouse game of illegitimate tools mimicing legitimate ones would come into play very quickly.

[sonofsanta]

I suppose the saving grace of the original article is the "in ten years" clause - after all, ten years ago we didn't have smartphones in their current form, let alone a successful implementation of tablets or workable online productivity apps (SkyDrive, Google Apps). So who knows where we'll be in ten years...

 

(of course, no matter where society is in ten years, schools will be at a level roughly equivalent to 2008 thanks to budgetary constraints keeping them a solid 15 years behind )

[Soulfish]

We already test ever year group online several times a year (online CATS, Reading and spelling tests - and yes it takes forever!), so not that much of a stretch to get to GCSE/A-Levels as well. To be honest as most have already said I suspect any form of online test taking will require a rethink in how exams are handled. Instead of the current everyone sitting the same paper I can see things moving towards having several batches of papers that are taken at staggered intervals and/or dynamic/adaptive exam papers that are different on each taking.

[gshaw]

Makes me glad to have done exams in ye olde style, OK for English \ History typing the long essay questions rather than writing may have been less painful on the hands but for anything else screen based is nasty. Trying to imaging doing A-Level Chemistry, Physics or Electronics on screen only would be a nightmare!

[mthomas08]

Don't know why anyones complaining, they will just do all this on their own ipads, it will all be fine with no issues.

 

In my experience there has yet to be a single online exam/test which has gone smoothly.

[sted]

Don't know why anyones complaining, they will just do all this on their own ipads, it will all be fine with no issues.

 

In my experience there has yet to be a single online exam/test which has gone smoothly.

 

yes their own devices complete with the i cheat app lol

 

i just see too many problems with exams on byod you cant check every file on every device and it wouldnt be hard to rename exam revision as say david bowie - heroes and sort of playing every second of every audio/video track you cant check

Edited September 23, 2013 by sted

[CyberNerd]

I bet in the 1830's the same arguments were held. Exams on PAPER! but the students could cheat!!!!

[Ephelyon]

And they do. But I suppose we've found ways of making that manageable by reducing the instances of it down to within what we've decided is an acceptable margin of error. I imagine something similar will happen if we now do it with tech; some cheating will still happen but so negligibly that we're alright with it overall.

[SteveBentley]

What about the scheduled lessons that take place....sorry ICT teachers, you can't teach your kids, we need your rooms for exams. Like that's going to go down well.

 

Compare and contrast with "Sorry PE Teachers, we're using the gym for exams"?