Key Takeaways

  • MIT engineering students now post near-100% scores on written homework by leaning on large language models, but their subsequent in-person exam scores reveal a sharp drop in real comprehension.
  • Justin Solomon, associate dean of engineering education at MIT, views problem sets as intellectual weight lifting where the struggle builds capacity, meaning outsourced answers eliminate the actual benefit.
  • To combat passive AI dependence, Solomon introduced short post-homework quizzes containing copy-pasted assignment questions to verify that students understand the work they hand in.
  • Advanced courses are pivoting away from static code submissions toward mandatory oral defenses and interactive presentations to ensure schools educate the student rather than evaluate Claude.

The Illusion of Frictionless Mastery

For decades, elite engineering programs treated completed problem sets as reliable proof that a student grasped linear algebra or differential equations. Large language models broke that proxy overnight.

“What we observe in the last year or two is that our students are getting nearly 100% on all their homeworks. Maybe it's because MIT students are super bright,” Solomon notes. Yet when those same students sit down for closed-book, in-person midterms without an active internet tab, the results crumble. Solomon observed the disconnect firsthand: “A lot of our students maybe perceive that they're learning when they use these different AI tools, but then they go and take the exam, and the score would say otherwise.”

When producing a clean derivation takes three seconds instead of three hours, students confuse recognition with execution. They read a model's fluent mathematical proof, nod along with the reasoning, and assume they own the concept. They do not.

The Weight Room Analogy

To change how students approach assignments, Solomon stripped homework of its status as an evaluative trophy. In his classes, a completed problem set carries value only as training resistance.

“In our course now, the way we structure it is there's still homeworks, but the analogy that I'm using, as with many of my colleagues at MIT, is closer to weight lifting,” Solomon says. “You don't do it because it's fun necessarily... But you do it more because you're trying to achieve some other goal, right? You're preparing or you're being a stronger person.”

If you hire an automated crane to lift 300 pounds off your chest, the bar returns to the rack, but your chest does not grow stronger. When students use Claude or ChatGPT to synthesize the hard steps in a proof, they hand in flawless work while leaving their cognitive muscles completely unworked.

Verifying the Human Behind Claude

Because faculty cannot easily outlaw AI tools, Solomon redesigned course mechanics to penalize hollow fluency. He added immediate, low-stakes friction right after submission: “We have a little short quiz after the homework that basically is just copy-pasted homework problem, but makes sure that the students at least comprehend what they did.”

If a student solved the problem set honestly, the follow-up quiz takes four minutes. If they generated the output without reading it, the quiz exposes them instantly.

For larger coursework, written reports are no longer enough. Solomon restructured student evaluations to focus on synchronous pressure. “We also had to restructure the project a bit, add an oral presentation component to it and bring some more interaction to make sure that essentially we're educating the human, not just testing the capabilities of Claude.”

What to Do With This

Audit your engineering hiring pipeline tomorrow. If your first technical filter is an unmonitored take-home project or an automated coding challenge, discard the written score. Replace it with a mandatory 15-minute live walkthrough where the candidate must defend three specific lines of their submission, explain edge-case failure modes, and refactor a single function on the spot without external tools.