Luis Hammer was in the middle of taking an exam for one of his online courses when his computer screen froze.
"Take off your Apple Watch," read a pop-up from one of the remote supervisors overseeing his test.
Hammer struggled to remove the watch. He then pointed his bare wrist at the screen and his examination continued.
He usedHonorlock, a remote monitoring program that uses a combination of artificial intelligence monitoring and live monitoring to ensure the integrity of remote exams. Honorlock is a Google Chrome extension that allows students to take proctored exams through Canvas without using a separate app.
"I was really surprised because I didn't know that people were actually watching and supervising me while I was taking the test," said Hammer, a sophomore who studies business data analytics. "I thought it was a computer doing it."
ASU added Honorlock as one of three options for monitoring remote exams in May 2021, a university spokesperson said in a statement. ASU also usesRespondus LockDown-BrowserEThe monitor would respondfor face-to-face classes, according to a universitydocument.
Respondus LockDown Browser prevents students from opening other browsers, printing, copying, pasting and searching the web. Respondus Monitor is browser-based, allowing students to record themselves using a webcam. Records are flagged for suspicious events and teachers can view records after review according to a responderInternet site.
Honorlock monitors live tests using AI to alert human proctors who delve into student exams via live chat to determine if they are violating academic integrity guidelines.
Honorlock offers several features to prevent fraud, mainly itspatented"Multi-device detection" to capture students accessing exam-related content on separate devices. It also offers a feature called "Search and Destroy" which removes leaked questions from the web and voice recognition to ensure that students are not collaborating with others during a test.
Students have raised concerns about Honorlock - specifically the pop-ins and the Multiple device detection function – online since the university announced its launch in Summer 2021, including on various Reddit threads.
OneGoogle reviews, Honorlock's Google Chrome extension has an average rating of one star out of 2,000 reviews. Complaints include concerns about data, privacy and general functionality.
In an email to faculty in May, the university announced that Honorlock would be used for iCourses, ASU online courses, and "high-stakes exams." substituteRPNow, a previously used monitoring service ASU.
RPNow had the students take a proctored exam. A supervisor would review the records and submit a report to ASU in accordance with abook pageno site RPNow for ASU.
"Each (supervisory solution) was selected through a rigorous process involving a committee of university faculty and staff and a comprehensive review of product functionality, user support and security," said a university spokesperson.
Under ASU's agreement with Honorlock, the university paid a total of $85,000 last summer for the program's soft start. ASU will pay $880,000 per year for three years, with an option to continue for a fourth and fifth year at the same rate. Total costs for a five-year use amount to almost US$4.5 million.
behind the castle
Teachers can choose which Honorlock features they want to enable for exams, so a student's experience with Honorlock can vary from class to class depending on teacher preference.
Nikolai Russell-Prusakowski, a student who started aPetitionagainst ASU with Honorlock and used it for a class, said he's more concerned about the multi-device detection feature, as Honorlock's own website seems to contradict ASU's text on how it works.
To use multi-device detection, Honorlock'sdata protectionsays it "does not scan other computers on your network or your phone or tablet". However, the Chrome extension has "the ability to detect alternate computers/mobile devices used to look up answers," according to a university website explaining the honorlock to faculty.
According to HonorlocksPatent, its multi-device detection works as follows:
- A professor sends exam questions to Honorlock, which they then "mark" by rewriting them with a combination of regular English letter and character substitutions.
- Watermarked individual questions are uploaded to Honorlock's bait sites called "Honeypots". When a user searches for a question by copying and pasting the watermarked question, the watermark causes the site to rank higher in the search results.
- Honorlock also creates a web beacon for each page. When the student's browser loads the page, the server hosting the honeypot sites logs a request. The request includes information such as:
- IP addresses, a sequence of numbers that identify devices or networks accessing the honeypot,
- URL,
- The time the page was accessed and much more.
- Honorlock collects similar information from the device the student is using to take the test.
- Additionally, every Honeypot website has an event listener, a line of code that collects user information including mouse clicks and movements, as well as the time the user spends on the website, which Honorlock can use as proof of fraud.
- If a student is using a secondary device during an assessment and visits a honeypot site, Honorlock will get the aforementioned information from the secondary device and the device using Honorlock and try to match them.
- Instructors can also ask a small number of unique questions - questions not found on sites like Quizlet or Course Hero - which are left unwatermarked and posted on Honeypot sites.
- Having unique questions that can't be found on other sites with possible answers increases the likelihood that a student will click through to a honeypot site and helps "ensure that at least some questions rank high (optimized for search engines)". simply retype (not copy/paste) the query into a search engine," according to the patent.
- One honeypot site, the Buzz Folder, has over 26,000 pages of blog posts with common test questions. When a student clicks the "Show Answer" button on a question, the site plays a sound that can also trigger the Honorlock's sound detection. According to the patent, "the attempt can be identified and presented later during testing as proof of fraud."
Information collected by Honorlock during an exam is gathered for the teacher, who can determine what to do when Honorlock determines that a student may have cheated.
Depending on the violation, students may have their grades reduced, fail a class, or be expelled from ASU.
Hammer said he had no significant problems running Honorlock scans, but said the program's ability to monitor your actions in real time "seems a little dystopian".
But using technologies like web beacons to anticipate people's activities is nothing new, said Katina Michael, a professor atSchool for the future of innovation in societyESchool of Computer Science and Advanced Intelligence.
"Honorlock is like taking advantage of what Google is already doing with us," said Michael.
According to a document providing answers to questions posed by ASU, Honorlock said that a benefit of his live supervision is that "it only affects his dishonest students, so honest students don't feel the punishment during the exam." live."
Tess Mitchell, Honorlock's senior vice president of marketing, said in an email that regulators often respond to actions that the program's AI beacons take, such as when someone steps on the screen.
"Honorlock Wardens can also proactively monitor any behavior that is not necessarily tracked/marked by the AI, but that violates instructor-defined testing instructions," Mitchell said in the email.
ASU provides teachers with writing suggestions to use on resumes to explain Honorlock. The recommended curriculum wording mentions an "integrity algorithm" and tells students "not to try and look for answers, even if it's a secondary device."
When asked how multi-device detection works, Honorlock spokesperson Mitchell said, "Providing an explanation of the process is not something we do, as such information could compromise the integrity of the tests," and referenced The State Press on the patent.
Teachers can also enable Honorlock's Search and Destroy feature. when activated,Teachers can upload exam questions to Honorlock, which then searches other sites for the questions and requests removal of the material by filing DMCA (Digital Millennium Copyright Act) takedown notices.
The risk of an “unlimited catastrophe”
Daniel Marburger, clinical professor at theeconomics faculty, said the need for supervisory solutions goes beyond the ASU classroom.
"When we first explored the idea of online courses, employers told us, 'It's extremely important for us to know that these courses and exams have integrity,'" Marburger said.
Marburger, who has used Respondus in his classes for about a year and a half, said that with nothing to ensure academic integrity, doing remote testing would be "an absolute disaster."
But David Boyles, an instructor atwrite programsin the English Department, said that by using automated supervision, ASU is missing an opportunity to better practice pedagogy.
"If your test can be easily fooled by someone searching Google, you've had a bad test," Boyles said. "You do a very superficial assessment that demands very little from you as an instructor."
Shelby Livingston, a senior business student, said her experience with Honorlock was perfect. In fact, she preferred to use him as a responder.
Despite the positive experience, Livingston said abandoning on-demand scans would be a more innovative solution than those that require close monitoring. The ability to apply learned knowledge in a real-world context that reflects a work environment may be most helpful to the student.
"I think knowing how to access that information, how to get the information right, is more important than having it in your head," Livingston said.
A more productive way to assess student comprehension would be to create tasks that are impossible to plagiarize or cheat, projects that go beyond just remembering, Boyles said.
"If you were developing better assessments and tests, or better yet, if you were moving away from those model bench tests that are about memory and having project-based assessments, things that really test students' higher-order thinking skills, you wouldn't have I did it." Problems of having to catch scammers and counterfeiters," Boyles said.
Honorlock as an impediment
Automated attempts to catch scammers aren't necessarily the most effective, said Patrick Hays, a graduate student studying materials science and engineering.
Hays, who once worked as an undergraduate teaching assistant at the Ira A. Fulton Schools of Engineering, said that in her experience, students could "easily" bypass supervisory software.
While Respondus, the monitoring program Hays worked with, flagged serious violations, it also flagged times when nothing unusual was happening.
"I know there have been a few cases where I've literally seen a person sitting there stiff and rigid looking at a computer screen, and during a 10 second gap, Respondus said there was another person in the frame and nothing; it was the same person," he said. Hays.
Likewise, Marburger said that about 99% of the flagged moments were wrong. It was the unmarked moments and the existence of the records that allowed him to perceive legitimate concerns about academic integrity.
Hays sees tools like Respondus and Honorlock as deterrents. They are not sophisticated programs of academic dishonesty, but blunt tools of violence, he said.
"If someone really wants to, they'll take your bike," Hays said. "But if you have a decent lock, it will discourage a lot of people from trying."
So far, it doesn't appear that the university has made an effort to make students feel safe about Honorlock, said Isabela Van Antwerp, a senior economics student.
If the university "could just be more transparent about everything to make sure any third-party browser extensions they use are super safe and secure," Van Antwerpen would feel more comfortable during exams.
By putting additional pressure on students on top of already stressful exams, teachers create a hostile relationship with their students, Boyles said.
"When you put all that stress on students, you're not really assessing their ability with the material," Boyles said. "They are assessing their ability to manage stress and deal with this invasion of privacy."
Contact the reporter atalcamp12@asu.eduand follow@Anna_Lee_Campno Twitter.
Asthe state presson Facebook and follow@statepressno Twitter.
Continue to support student journalism andDonate to State Press today.
similar posts
The ASU Student's Guide to Filing Federal and State Taxes Before Tax Day
VonSadie BuggelERichard Parker
Sun Devil Motorsports unveils 2023 Formula SAE racing car
VonLauren Boykins
Tour of ASU's CXFEL Facility: X-Ray, Lasers Up Close and Still Ongoing Research
VonAudrey Eagerton