Analyzing student votes across AI models for college essay help

Join the discussion

• Hacker News
• I'd love to see the same experiment with responses normalized for length, and with actual essay quality scored separately from how helpful the model's feedback felt
  by Honali
• Will give this a shot - we're going to try to work with students and schools better on normalizing this stuff out
  by pasharayan
• > “The best response must be complete without becoming shapeless, structured without sounding mechanical…”

  Anyone else scratching their head at these descriptions? Uh, your writing is shapeless. You’re 5% too mechanical. Who talks like this? Fiction writers and editors??

  by xtiansimon
• TBH :') that one was an AI slip through. In other comments i've flagged the things that were hard rewritten - but even on me that's bad.
  by pasharayan
• True. Whenever I ask LLMs to translate English to other languages, Gemini is always the most natural one.
  by hesprs
• Been using Gemini for about a year for code (much cheaper and actually sticks to your prompt so savings are many multiples of; I work with Salesforce implementations and Google is a user for Salesforce so I think their code might have went into training data, coupled with their code guidelines it's been a bliss for me) and for other uses for maybe 9 months.

  I do go back to ChatGPT for things that need to be calculated, but _Gemini simply hallucinates less_. IMO that's whats important for students.

  I did get suckered into extending my Grok subscription which is really fun for images/videos. Grok in car is a must too (my jaw dropped when I asked what is pantsula music and it added multiple albums into playlist for me).

  by dzhiurgis
• I use Fable for writing code and some planning. I use GPT for planning/roadmaps and code review, it checks Fable. I use Gemini for stray conversations. Gemini has a better conversational style than GPT, less robotic; it hallucinates more unfortunately, it has plainly fallen quite a bit behind, but it's still highly useful to converse with, research with, etc. And it spares me wasting usage of GPT or Fable. I use Gemini for the 'everything else' category, basically. Gemini never goes anywhere near anything serious.

  Fable & Claude Opus 4.x or 5 are terrible to talk to about anything. I gave up on that entirely and just use Anthropic's models for work.

  Any less serious technical work I'll use GPT for, as the usage limits are quite fantastic.

  by adventured
• Thankfully, college essay writing isn't in that 'serious' category!
  by pasharayan
• GPT gets the most out there, but creates interesting nuance that turn an idea into a more creative exercise. Claude is like a more grounded GPT but may miss the nuance. Gemini is my last pick, but does better than the rest for making something clear and understandable. It gets to be exhausting parsing through Claude outputs and breaking it down into something more easily understandable. One way I use to improve this is to ask the LLM to pull upstream ideas from prior work on the topics. This makes me feel better about the possibility of hallucination, gives me alternate places to look, but the model still may pull things out of context or fall over on the interpretation.

  Given all that, I can definitely see how Gemini would be preferred. And good for Google, because I would rather my offering be the top choice for the most people rather than better serving a small subset of users.

  by gexla
• Gemini is bad at coding, but good at writing. More human like
  by ShawWang
• Makes sense when you consider what type of data Google had available in abundance. Natural written language (e.g., Docs, Gmail, Books) and natural spoken language (e.g., YouTube).

  One could argue that Google Colab would supply the training data for better coding performance. I would argue that Colab is mostly used for non-complex (e.g., small number of variables) and self-contained (i.e., runnable in one page) code that can’t train a model for multi-folder and multi-page projects that rely on global connections, which real-life coding would often require.

  by madhatter999
• Gemini is not bad at coding? Give it a /plan and it does a very decent job in a fraction of the time/tokens that Claude needs.
  by parasti
• Anything is better than the current crop of Claudes, its prose has become painful.
  by gherkinnn
• Fair challenge, the ability to write good prose is load-bearing for essential tasks.
  by asdfologist
• yeah i think Claude might just be compounding its way to a local maximum rn
  by whateveracct
• It's so bad that my workflow has Sol rewrite all comments and error messages produced by Claude.
  by senderista
• You make some good points, but the best one is the one you might not be aware you've made.
  by purplecats
• For real, the "Claudish" has become so painful to read that it just takes me out of whatever task I was working on.

  When prompted to use simple English without jargon, it's still filled with load bearing honest caveats in every footgun seam it talks about — what I should have led with. <Insert whatever other Claude cliche you prefer>

  And I'm not the only one to notice this. Next time it's up for renewal, my team is abandoning it for GH Copilot in order to use literally any other frontier model.

  by StableAlkyne
• In my experience, Gemini excels at explaining new concepts clearly and intuitively. Much more so than other models.
  by John7878781
• I wonder how much is due to their investment in notebook LLM + gmail work
  by pasharayan
• So, given that developers these days mix and match models anyway with harnesses like Pi, has anyone tried letting Claude or GPT do the coding, and Gemini do the comments and documentation? Maybe even asking it to "translate" Claude's output messages to better human language for the developer or subagents?
  by inbx0
• Gemini is the best general-purpose model. We hear a lot about the other ones here because we are focusing on coding.
  by zorked
• > How to use Gemini for a college essay

  Given that this is increasingly the go-to for a college degree, college needs to rethink its cirricula and place in the world. Or at least get rid of the essay.

  Lest it become a place where student and teacher ais go to play pay-to-win social deduction video games.

  by avaer
• The homework essay is dead. High schoolers in my home state were just relieved of any take-home assignments because it was pointless now, I guess universities won't be far behind.
  by brainwad
• I think it's less about curricula and more about how professors should eval their students. In previous era's essays are a proxy of the students ability to reason, remember and argue certain points - but its clear they fall short now.

  I suspect the best schools into the future will integrate lots of socratic defenses of theses, building real things in real time or solving problems with a professor in a case study manner. My last company was a good hint to the future.

  by pasharayan
• My professor and teacher friends are painfully aware of how many students submit AI generated essays.

  They're adding steps like having the students discuss and defend their essay, which immediately reveals the people who had AI write something and thought they could bluff. This triggers complaints about social anxiety and such, which are unfortunately becoming the go-to defense when unable to discuss the work.

  They're also moving toward more in-person writing. Instead of long essays, shorter writing segments as part of the test. Submitting a written essay earns you feedback from the professor and a better understanding of the topic, but that's it.

  by Aurornis
• Author, that doesn’t mean you need to get Gemini to write your article.
  by demibabs
• On the bright side, at least it wasn't Claude.
  by userbinator
• TBH i did a bit of both - we used AI to get us insights in the db, we then wrote and rewrote the article to be readable. It's really hard because usually when i write reports they're so terse its not fun to read
  by pasharayan
• The spread isn't as large as I thought from the headline. It's closer to a toss-up than I would have expected.

  The length of the response is a huge factor:

  > Students tended to prefer longer responses. The selected answer was 37% longer on average than the alternatives. The longest response won 47.7% of decisive writing comparisons. The shortest still won 25.0%.

  So the score is partially a proxy for longest responses.

  Makes me wonder how much the reviewers actually read the text. Were lazy evaluators picking the text that looked the longest or most structured without reading it all?

  by Aurornis