Most standardized tests have a fee: the SAT costs $50, the GRE costs $200, and the NY Bar Exam costs $250. This year, the bar exam came at a much larger cost for recent law school graduates — their privacy.
Many in-person events have had to find ways to move to the internet this year, and exams are no exception. We’d like to think that online exams shouldn’t be a big deal. It’s 2020. We have a pretty good grasp on how security and privacy should work, and it shouldn’t be too hard to implement sensible anti-cheating features.
It shouldn’t be a big deal, but for one software firm, it really is.
The NY State Board of Law Examiners (NY BOLE), along with several other state exam boards, chose to administer this year’s bar exam via ExamSoft’s Examplify. If you’ve missed out on the Examplify Saga, following the Diploma Privilege for New York account on Twitter will get you caught up pretty quickly. Essentially, according to its users, Examplify is an unmitigated disaster. Let’s start with something that should have been settled twenty years ago.
Did They Just Email Me My Password?
Passwords are stored in plaintext. Seriously- how is this still a thing? Users report being able to call customer support and retrieve not only their usernames but their passwords as well. Others had their passwords emailed to them. If a customer support rep can read your password to you over the phone, you’ve got a real problem. It would only take a bit of social engineering for somebody to get into your account, and if you reuse passwords, then your unrelated accounts can be easily compromised. (Please, use a password manager!)
It turns out that you don’t even need to log into somebody’s account to scope out their personal info. Users uploaded government IDs to their accounts, which were promptly sent to a server and stored accessible publicly via a random URL. It’s worth noting that it seems like more of a shortcoming on the part of the NY BOLE and not ExamSoft. Thankfully, once users reported the problem, the BOLE fixed it — though the fact that it was a issue in the first place is ridiculous. More astonishingly, the NY BOLE isn’t the only exam board that had this sort of problem. The DC Bar published users’ background check documents, containing SSNs and employment histories, in addition to their IDs.
12345? Amazing, I have the same combination on my luggage!
There is a set of config files that comes bundled with each downloaded exam, and those files are, well, just plaintext. Puzzlingly, the parameters in these files, such as “isTimed” and “allowSpellChecking,” seem to have little effect in the software. Examsoft claims that modifying them will corrupt and invalidate the exam, but that doesn’t appear to be the case either.
The software also downloads the exam days before the test starts, leaving the files vulnerable to being poked and prodded by the curious-minded. These files are at least encrypted with an 18-character key, according to ExamSoft. On the exam day, the key is made public and test-takers can use it to decrypt the exam files. This isn’t the best strategy, especially when the Michigan bar exam, rather than employing an 18-character key as ExamSoft promised, used the passwords green56, purple34, and blue78 for their exam files. These horribly weak passwords are probably vulnerable to a brute-force attack.
One of Examplify’s main selling points is its ability to lock down a computer. You wouldn’t want a test-taker googling anything, or glancing at a PDF of the textbook on a second monitor. While some of the more obvious ways to cheat are blocked here, users have found a loophole that ExamSoft unsurprisingly overlooked. Mac users may be familiar with the Universal Clipboard, a feature which, when enabled, allows a user to copy something on one device and paste it on another linked to the same account. Guess what — Examplify doesn’t disable it. This is a fun loophole, since a test-taker could copy the question text, and a friend could paste it onto an iPad and copy the relevant section in the textbook for the test-taker to read. It’s unclear as to whether or not this has been fixed yet, but as of September 28th the workaround was still usable.
Users found other creative ways to get around the software lockout as well. It’s been reported that if the computer gets rebooted mid-exam, the user gets ~30s of unrestricted access to their files, as well as the internet. One user was even able to reset the timer and start the exam over by rebooting their computer.
As you can imagine, it’s not just the security that’s awful. Users report a slew of interface problems, from software freezing mid-exam to the facial tracking software (which determines whether you’ve cheated by looking away from your screen) not being able to recognize dark-skinned people.
Now What?
It’s tough to dive into this sea of shortcomings without registering for the bar exam, however we were able to find a link to the Examplify installer (Windows, macOS) hosted on ExamSoft’s servers and, of course, unprotected. Go nuts.
It’s impressive that with modern advances in technology, we’ve found ways to make exams more stressful than ever before. It may be worth considering doing away with the bar exam and other standardized tests entirely. For the time being these tests are still a reality and it goes without saying, but we’ll say it anyway: even though ExamSoft and the NY BOLE seem to have made it easy to cheat, please don’t do that. Amid all of this failure, they’ve at least succeeded in one way. They’ve given us excellent examples of how not to do, well, pretty much everything.
Thanks to [Jonathan Merrin] for the tip!
When such systematic failure is encountered at some point the question has to be ask…is this deliberate?
Well, it IS New York State…
This seems terribly misplaced: “Please, use a password manager!” Firstly, it would be better to recommend passphrases. Passphrases address some of the problems with human memory, as well as making more secure “passwords”. The human memory tends to fail at seven units, which was why phone numbers (without area code) were seven digits for so long, and why when phones started able to store numbers in memory, the 15 digits (17 with area code) didn’t become a big problem. And, it seems like you’re promoting a class of products. This is like recommending VPNs for privacy, when using one actually puts users on blind trust and dependence of VPNs which might not actually safeguard users’ privacy at all. A bad, or incompetent developer of any particular password manager negates the benefits, absolutely. Secondly, it isn’t relevant or pertinent to issues with Examplify, so there’s no reason to bring it up. I was inspired by the thoroughness of the article on the subject, so it likewise deserves some thorough constructive criticism.
Except the line your complaining about comes directly after a note about password reuse, which is very relevant to password managers.
Are we really talking about a group savy with computers? Who’s going to be “brute-forcing” anything?
That’ll be us patent lawyers with degrees in EE/CE/CS, ’cause we ain’t smurt enough to pass the bar exam without some help.
I once met a Patent Lawyer with a Mechanical Engineering degree.
His isometric drawings were GOOD!
I used to do isometric drawings freehand with just a ruler and pencil. I was able to perfectly eyeball 30 degree angles. Why? Just for the fun of drawing stuff, and because I took drafting classes in Jr. High and Highschool. Use a T-Square and triangles enough and you can learn the angles etc to where you don’t need the square and triangles. CAD? Didn’t *have* CAD back then, at least not outside any large company.
Money can be exchanged for goods or services.
A yes, configurable config files. The irony. Anyone with enough knowledge to prod around the software could break this thing. I didn’t know it had facial recognition though. Interesting.
Even trying to lock down a computer is pointless. One could always simply get a second computer (OK Google,. what does habeas corpus mean?)
One possible solution would be to require the user to position a webcam behind and above so the camera records the complete user test session in enough resolution that questions and answers are visible in the replay.
and/or you could have a combination of realtime and recorded such that a set of witness could switch between webcams and watch the actions of the test takers during the test. Detecting cheating shouldn’t be that difficult.
Until the test taker streams a prerecorded video as the webcam feed. doh!
There was a bit of controversy over software like this (I don’t know if it’s exactly the same system, I can’t remember) just completely shutting itself down and claiming you were cheating if you have a virtual web cam driver present on the system (to hell if it’s the one actually feeding video to the software). Lots of game streamers had big issues come exams in lockdown.
Wouldn’t it be cheaper to buy plastic dividers and have the exam in an open field somewhere? It doesn’t need to be indoors, or packed tightly, or purely digital.
Ah, yes, but the digital solution becomes so obvious when there is sufficient brib… lobbying.
Plus, the people making the calls likely have no grasp of IT security, so a powerpoint showing how cheaper digital it is compared to renting a room, supervisors, security guards, sanitizers is likely to sway them over anyway. Ever since 2008, the argument of cost reduction succeeds over any other rationale. Much easier to explain to your superiors or voters.
“Go nuts”? “Go nuts” as in “illegally gain unauthorized access to a computer system which does not belong you”?
I laugh at your $250 dollar test fee. Every exam I ever took (US medical) required a test fee well in excess of $1000 (one was $2100 and required travel and stay in “recommended” [required] hotel). The ones not requiring travel were still in the neighborhood of $1G, and additionally required a test-site fee (Thompson/Prometric) usually about $200 on top. Not to mention license fees of $850 every two years, and, oh, if it happens to be Texas, a “registration fee” of like $500 for your already expensive license fee.
BTW the security at the test site was pretty good: full body frisk, metal detectors, fingerprints… we even had to turn our pockets inside out. Lord Jebuz if you had to take a leak in the middle- you had to repeat the entire process (on the clock, of course) to come back in.
Pee bottle permitted? Don’t tell me a bunch of doctors are squeamish about seeing someone urinate. =D
How should a test fee in the range of 1G (= 1E9, 10^9, a billion, probably US$) possible to finance in a lifetime? I do not know any other meaning of the numeric prefix “G”.
1 “grand” = 1000
Cameras spot you looking away? This seems heavily biased against people with ADHD, ASC, people who sit back to think, anyone who glances up to aid recall, or, you know what, most people…?
Also, what kind of software is it if it locks stuff down like that?! Sounds like they’re rootkitting your PC. I really hope they’ve done it properly and not left any holes for something else to exploit.
Wait a minute, eye tracking? Seriously?
1) My PC does not have a webcam
2) try to find my eyes in the eye spam of doge hat, doge T-shirt (got them on ali-bay) and doge walls.
3) did i mention my lamp broke and the webcam is the half-dollar chinese dust without LEDs and with settings done by my cat?
Having used Examsoft, its built around the premise that the exam taker and the area around them is being monitored by a proctor. So this all doesn’t surprise me.
Typically bar exams are done in a large convention center with rows and rows of tables, you go in with your laptop, 3 pencils and scratch paper and they monitor and time you independently from the software.
Don’t want me looking at another screen huh?
*flexes PiP function on 20 year old monitor*