Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- The last take-home test I did was for EE364a: Convex Optimization. It was a 24 hour test, and I had a cold. I booked a hotel room as my apartment didn't have air conditioning. It was brutal. I got most of the programming questions correct, but only a few of the proofs. The average of the class on this test (and most every other assignment) was 80%+. I got an A- overall in that class.
Perhaps this class isn't too hard for Stanford students, but I have to wonder about cheating when the averages on nearly every single assignment is that high. One clue was office-hours: Whenever I dropped by office hours, there was always a line (didn't matter the class). I quickly learned that TAs would often drop a hint that would make particularly hard problems easy to complete. It provided an unfair advantage to students who could attend office hours.
It also reminds me of the huge scam of cheating on the USMLE amongst Nepali medical students: https://www.medpagetoday.com/special-reports/features/113627
I've met many wonderful international medical graduates. Many have shockingly high USMLE scores. It is true that there's no time limit on their preparation (U.S. students have under 2 years to prepare during medical school, international students may wait years after graduation before taking the exam). Before that scandal, I never would've thought someone could cheat on the USMLE. Prometric test centers are crazy locked-down. But that's not how they did it. They did the long-game. Prior test takers would remember a handful of questions and just add them to a secret database. After many years, that database contained nearly every question on the exam. Test takers would work extremely hard - memorizing every single question. The reward of a U.S. residency is life-changing, I get why cheating was(is?) so rampant.
by aabajian - One thing I would say from the student perspective is, when you know for a fact that your fellow students are using LLMs, and you feel like you can do it honestly and earn a B, or use LLMs and get an A, it msut be a tough decision. Grades are often adjusted on a bell curve and honest students are disadvantaged when others cheat. In my undergrad I was leading in nearly all of my classes for 7/8 semesters, but the final semester had online exams due to covid, and I was suddenly a barely above average student in the last semester.by haritha-j
- His research is in Game Theory. He should have realized that, in a situation where all competitors are (possibly) using LLMs, the game theoretic optimal choice is to use LLMs.
- When you're a student in a competitive program at a top university, graded on a curve, and you know your fellow classmates are cheating with AI, you have little choice but to do the same. Especially when jobs for new grads are harder to come by and there's more pressure to also go above and beyond with internships and side projects during your time in school. There's no way to compete without cheating.by pants2
- > When he reported the case to high-ranking officials at Brown, he got a cold reaction. The response from the president, he said, was absolute silence. The dean did not comment either until Serrano took the case before the Academic Code Committee. At that point, he received a note acknowledging that what had happened in his classroom was “a wake-up call.” Serrano, a Madrid-born economist who has been at Brown for 34 years, believes this is not enough. “That cannot be the university’s position before an incident of this magnitude. Academic integrity is a value worth defending. The faculty cannot be left on its own
Is anyone surprised that the academics are essentially being left on their own about is? University management has become increasingly disconnected from the academics doing the work on the ground at least during the last 20 years that I've been in academia. Covid was really an eye-opener in this respect, at our university academics were told with very short notice to make all lectures remote. There was essentially zero support from central management, there was not even budget to buy headsets or cameras and academics were expected to use their research funds.
At the end of the pandemic academics were thanked with two years of 0 and 1% salary increase, due to a tough economic situation. During the same time the president received a 8% raise. The tough economic situation was entirely due to mismanagement, i.e. the university had invested pension funds into fixed interest investments which did not keep up with pension growth (during a time when stock markets were going from strength to strength).
by cycomanic - I have seen it firsthand in the CS department here at Dartmouth. It is bad.
We're currently designing a new intro systems curriculum, and we're thinking of it as an adversarial problem. That is, we're designing the course to ensure that a student optimizing for the best grade per unit work still meets our learning objectives. That means, as everyone else is saying, paper exams, but also 1-on-1 interviews to check that students understand each assignment they turn in. These interviews feature both factual questions ("You're using this macro from that library. What does it do?", "Please describe what this function does and how it works.") and conceptual questions ("Why is this code structured this way instead of $whatever?", "How else did you try solving this?", etc.) This doesn't stop students from generating code, but at least they have to understand that code in detail.
This is not as good as writing the code yourself, but how much worse is it? For math classes, this gap is gigantic. Obviously, understanding someone else's proof is much easier than writing your own. For programming classes, I think (without evidence) that the gap is somewhat smaller.
My experience from the past is that when this kind of evaluation is made clear up front, the students know what to expect and either do fine or drop the class in the first week. If you start with take-home exams and then spring paper exams on them halfway through the course, then half the class is cheating and won't be able to recover, as we read in the article.
In general, our students are somewhat motivated by an abstract desire to learn, but are much more motivated by grades. If there exists a straightforward path through your course that leads to a good grade without doing much work, most students will take it. (Our undergrads' course review website is literally called "Layup List." They are actually this shameless.) It's our job as instructors to ensure that all paths leading to a good grade either require learning the material or are more difficult to pull off than just learning the material.
It's best not to blame the students. They are good at optimizing metrics; that's how they ended up here in the first place. We just need to better align the evaluation metrics with the outcomes that we're looking for.
by bkallus - I was born in 1970. My teenage years were in the 1980s, in coincidence with the emergence of the home computer. I had a Timex-Sinclair 1000, then a C64, then a C128. One of my main forces that drove myself to learn computers was (besides games) the ability to produce text that was clearly written and readable, thanks to word processors and printers, because my handwriting was abysmal, and I got tired very quickly while writing by hand. In the computer, instead, I got to type exactly what I wanted, I could edit and correct my spelling and grammar mistakes, and then produce very readable and clean output. A win in my book.
Now, I require of students to submit written assignments done by hand. That way, I can at least be certain that there's some learning involved, even if they resort to AI to produce the relevant written part, because evidence points out that writing by hand reinforces learning.
I read that other professors resorted to requiring manual typewriters, which I also hate with a passion.
That is, AI is negating decades of enablement achieved by technology.
Honestly, between these circumstances and the fact that we are "enjoying" (?) brave new prices for RAM and SSDs, I'm finding AI increasingly harder to like.
by sombragris - In the AI era, tests are going to have to be in person and hand written.
I have written an article on how I have adjusted my classes to the situation:
https://htmx.org/essays/universities-and-ai/
Ironically, I think the AI era may make university degrees a better signal of the intellectual abilities of students due to the presence of pre-computer infrastructure like large lecture halls, industrial-scale copiers, etc.