Using Zoom to proctor an exam remotely

I had two brave souls out of 7 registered for the re-sit exam in Informatik 2 (Algorithms and Data Structures) on March 30, 2020 in the middle of the Corona Lockdown.

I set up the Moodle exam as normal, which is 90 minutes of test time with 105 minute window of the exam being open. This helps overcome anxiety at "missing out" if something goes wrong. Everyone gets 90 minutes, no matter when they start. I also set a password for the exam that will only be communicated to registered exam takers. I used the JavaScript full-screen security, because I can't have them install the SafeExamBrowser.

On the advice of Tricia Bertram Gallant, an expert on remote academic integrity from the UCSD Academic Integrity team, I decided to use Zoom for proctoring. There are issues with this, as Zoom was not GDPR compliant at the time. They changed that around April/May 2020. But in this emergency situation the exam was declared to be voluntary, they can retake it later with no penalty if anything goes wrong.

I offered a preparatory exam with silly questions the day before so that people could try out their equipment. We discovered that Kapersky Anti-Virus wouldn't let Zoom access the camera, so that had to be fixed. One student could not get his system to remain stable, his browser kept crashing, so he decided not to take the exam. In the school labs I have complete control over this, so one possibility might be having the students use a VPN into school and use the school computers via remote desktop. That does, however, generate a LOT of Internet traffic.

You have to enable breakout rooms in Zoom, that is hidden deep in the Administration panel under Account Management /Account Settings/ In Meeting, you must enable screen sharing for participants and breakout rooms.

Create a meeting and make sure you have a password so that people who can guess the meeting number can't drop in. Communicate this password and the Moodle exam password as well as a description of how the exam will work by email at least 45 minutes in advance. Advise students to be in the Zoom room 15 minutes before the exam so that you can greet them and explain again how the exam will work. Make them do stress-reducing neck exercises :)

At 5 minutes before the exam time in Moodle I requested that Zoom set up one breakout room per participant and automatically assign people. This was very fast, and when I clicked on the button people were immediately redirected there. They started screen sharing, and I could check that everyone was in the right place for the exam to start. They were nervous waiting for the time to begin. One woman was using Edge, it was not behaving well, only showing that the exam was not on yet, even though the time was right. I set the begin time in the Moodle back 15 minutes, and now Edge finally gave her access to the exam. No idea what caused that hiccup.

I then cruised through both breakout rooms (this would be difficult with 40 students!) to make sure everyone was started with the exam. One wasn't, she had not written down the password for Moodle properly, even though I said that they should print out the instructions I sent. I gave her the password, and she was able to get started. I made sure to smile when I entered the room so that if they saw my video they wouldn't be irritated. Mostly they didn't see me at all. [It turns out there is an "ask for help" button that only they see. It is really great, it opens a window for me to just click on and I am taken to that breakout room.]

I also checked in Moodle to make sure that only these two people were taking the exam.

The problem was that I didn't see any possibility for them to ask for help, that is, some sort of hand raising that is available in the main meeting room. So I went to check on them a bit more often (every 10 minutes) than I would have normally. The idea is that I can drop in at any time – if they are not seated at the computer, if there is anyone behind them or talking with them or anything unusual, this would be an automatic fail. I just select the room in the Breakout Room window and I am popped in. Sometimes they did not see me at all, other times they noticed that I was there, but didn't say anything.

Some sort of possibility for reaching me would be really necessary, perhaps that is the "Remote support" thing that I could have changed in the options during the exam, but I was afraid that the connection might be broken so I didn't want to experiment with this while they were working.

I would also like to be able to see their screens without them seeing me :)

It was interesting to see that both woman were talking a bit to themselves as they solved the problems (or in one case shook her head: "No, I don't remember that"), something they can't do in an exam room. I wonder if students do better on exams when they are able to speak like that instead of having to keep quiet. Research question!

One woman finished early, she decided the exam was too difficult and decided not to have it evaluated. The other passed and was so happy to be able to check off this course from her list.

What would I do differently?

• Enable remote support
• Turn off chat for this meeting [It turns out that in the breakout room they only see stuff from the main room when they were there and their local chat, so that's fine. ]
• Have them use their orthonyms (real names) not screen names
• If they register with Zoom it would be easier for me to restrict the room to only registered students, but that is a GDPR issue. With me allowing them to use screen names they are not sending more information than their IP addresses and browser information to Zoom.
  

What was difficult?

• I was not able to check that they only had the normally permitted number of "cheat sheet" pages (2) that I normally permit first-year students to have. Actually writing the cheat sheet by hand is the best device for having them review the material!
• Normally during the exam I surf or grade papers, looking up and around the room every so often. I felt that I couldn't do this, even though they could not request remote support, as I needed to be visiting them every 10 minutes.
  

What did the students think?

• One student wanted a calculator.
• The other wrote:
  
  "Zoom has proven to be very user-friendly and easy to install. The separate breakout rooms are a good additional function, especially for exams.
  
  I was a little nervous at the beginning about the proctoring, but after a few minutes I were more focused on the tasks.
  Because I had minimized the small “camera” window showing up during the screen transfer, I did not notice when you entered the breakout room, therefore I was not distracted when you stepped by now and then.
  
  I think because some of us students are already used to writing the informatics exam in Moodle, there will be no further difficulties here in the future.
  
  All in all it was an interesting and positive experience."

Update 2020-05-12

I ran another experiment with Zoom proctoring today with three students. One of the students ran into technical trouble, his Windows machine insisted on an update. I know this from Moodle: don't panic, it saves your work. I saw him disappear from the breakout room, he eventually re-appeared in Zoom, panicked. I calmed him down, put him back in his breakout room, had him share his screen so I could talk him through it. He logged back into Moodle, clicked on the exam, and was soooooo relieved to see that it had indeed saved his work and he still had plenty of time.

An issue that bothers me is that I have a dual monitor setup, I want the breakout rooms to STAY PUT on the second screen. But it keeps dumping me back on my primary screen when I get out of the breakout room, and I have to click on it again. I just want it pinned to the screen, just like I want the chat to be pinned and not disappear when I start sharing my screen.

Update 2020-07-14

I took a deep breath and ran a full-blown exam via Zoom. I decided to make it open book, as I wanted them to use external tools to draw UML diagrams and have them upload the files to Moodle. That won't work in the secure browser, so I had to keep everything open. Which was fine, I wanted to experiment with open book exams anyway. The external drawings worked very well - some drew on paper and uploaded a picture from their phones, others used various tools and took screenshots. I gave 15 minutes for this, and it was sufficient for most to complete the question.

I asked people to be there at least 15 minutes early. Checking the ID for 34 people is a pain in the backside. I wanted to take people off to a breakout room for privacy, but that took far too much time (at least 2 minutes per person). So I said that anyone who didn't mind could just hold up their ID to their cameras, and I would check those left by cruising around after the exam started. Most held up their cards, I could actually read their names and see their faces, the quality is good enough! I had two people without cameras, I had them use their phones to take a photo of their ID and upload it to their computer while I watched. That worked fine.

Bizarrely, it looked to me like 10 of the 34 breakout rooms didn't have people in them. I checked Moodle, 34 were writing. I took names of those in Zoom and wrote an email via Moodle to the 10 who were not there. One guy saw the email right away and had me come to his breakout room - he was indeed still there. When I got back to the main room, all 34 people were cozily in their individual breakout rooms. Must have been a bug somewhere. The mantra I tell my students is for me, too: Don't panic!

I asked the students for feedback after the exam:

• Students who do not speak English as a mother tongue were so happy they could translate the questions with Google translate into their native languages.
• Some asked for permission to listen to music - that was fine by me.
• One complained that it was impossible to Google for the answers to the questions :) Yes, I designed all but one of the questions that way. The last question was, however, copied from the textbook we used. There are any number of answers available on the internet. I made a note of the URLs of the first three hits in Google in the grader's notes in Moodle and checked these. I was rather disappointed that a number of people did copy these answers without reference, although I specified at the beginning that they were not to copy & paste and they were to give their sources if they were using outside ones.
• One noted that he really had to think about the questions asked.
• Another noted that re-reading the textbook in the week before the exam was actually beneficial.

Would I do it again? Absolutely! But only as open book, trying to proctor all these people would only drive me crazy. Oh, and I had different exams for everyone, Moodle has a neat feature that lets you put together similar questions and then Moodle will randomly assign the questions to students. I had 8 different requirements that they were to evaluate, and had Moodle choose 3 per student. That helps make it hard to collaborate via WhatsApp or so...