This is already the case for Master's degrees and above in Denmark. I've been the opponent examining students. We ask Master's students to give US instruction on a topic (drawn randomly on a card in front of them, from a list they know a few days in advance). They do a chalk and talk on a topic for about fifteen minutes with three to five professors being "dumb students". It works fantastically and at the end it's very apparent to both student and teachers how well a topic is understood. Marks are rarely contested and in my experience most students think the process is fair.
Of course, woe betide those with social phobias....
I worked in sales for a few years when I was younger before I started my actual career, I went from an introvert afraid of public speaking to completely comfortable breaking the ice with basically anyone in those years (mostly because I liked having four walls and a roof) and that led to me doing technical training/talks to strangers years later.
I still don't really like it but I also realised that dragons don't appear from portals and eat you immediately if you do it.
My missus is much like I was but she never got through it, she finds it fascinating that I can.
The social model of diability is great in some ways (encouraging activism), but the idea that everything needs to be changed to be "designed for" people with disabilities has its downsides. Not all disabilities can be overcome, but to the extent that some can (at least partly), often IMO it's a good thing to have an expectation that most people can deal with the real world.
I was good at it because I sold the customer what they needed vs what the company wanted me to sell them, it actually led to the same customers coming back and asking for me by name and I got offered a job by one of them - I do wonder what my life would have been like had I taken it but it also led to my entire career as well, my first consulting gig came off the back of a conversation with a customer about a quote they'd had for some development and with the hubris of youth I said I could do that way cheaper (I could and I did, found out later how much cheaper and kicked myself, also a valuable lesson - You don't get paid for your effort, you get paid to save them money, if you are saving them a lot of money you can charge a lot of money).
Was never going to be my career and didn't pay particularly well so I wasn't too concerned about what management thought long term.
I studied my masters in Denmark 5 years ago, before AI. This was absolutely the case - first oral exams in my batchelors many of my fellow students were incredibly nervous. By the end the exam atmosphere was very chill. I definitely improved with public speaking, and so did the other students - including some who had been almost incapacitatingly anxious (throwing up, requiring a lot of convincing from us to even attend exams) and ended roughly the same as the rest of us
What also helped me is being so knowledgeable about the topic that I didn't need to prepare the presentation much (except for the slides and the general plan). When you know what you have to talk about, it becomes a lot more natural and relaxed for me.
I would be a lot more stressed if I had to learn a presentation by heart and it would probably be a robotic delivery.
So I did quite a few "surprise" presentations, with very light prepared material and a topic I cared about. Got me the practice I needed and I learned not to get "stuck" as if I forgot the next sentence. The presentation always flows!
Now, I have a talk at a major conference, and I'm afraid this won't work there, so I'll prepare my talk better this time, maybe even have speaker notes!
That's part of what fixed it for me. What really fixed it was three things:
1. Actually being very knowledgeable about the subject in question, to the extent that I can offer facts or well-reasoned opinion on the topic on essentially any reasonable question. In other words, not being a junior high school student asked to do a 15 minute speech on a topic drawn from a hat.
2. Not having the idea that the audience expects a correct and immediate answer to every possible question.
3. Not doing every single speech literally in front of 15% of my entire peer group, including all my friends, all the people I admired, and all the people who treated me poorly.
I still hate doing Zoom calls without slides or a screen to share because I can't stand having 20 people staring intently at me. It's just distracting. But otherwise I have no problem at all talking for hours on a topic I know well.
Likewise, I thought I was bad at public speaking and avoided it almost entirely until my last couple years of university. By then I found myself surrounded by friendly, curious peers who were happy for me to nerd about whatever it was I became a semi-expert on. That confidence served me well once I needed to lead/train a team at work.
Doing a presentation on Zoom seems kind of pointless as you lose the connection to the audience. I'd rather just assign a pre-record to watch and do a live Q&A later on. More like a Lecture+Tutorial format.
One thing doctor, and to a lesser extent lawyers, have to do is project confidence when they may in fact not have much. A doctor/lawyer regularly has to quickly become an expert on a subject, or seem like one, often overnight but sometimes much faster than that. Briefing something you are still new to, without getting it horribly wrong, is a skill that must be developed.
I run into this in the military too. "The person to do the ethics/roe/safety/drugs/mental health brief just called in sick. Two hundred soldiers will be here at 0800 tomorrow. Read fast."
As someone with lifelong social anxiety. There is the concept of White Knuckling an event. If the anxiety is so far beyond your capability that start to panic, you only reinforce the fear instead of soothe it.
That’s why exposure therapy with an anxiety ladder is important
Right, that’s the problem: for most people this works, but those it doesn’t work for, it can really fuck up. So what this amounts to in expected utility is you need guardrails before forcing things this way.
The best class I ever took in any school setting was Public Speaking in my first year, first semester in college just to "get it out of the way."
The teacher started off with "you're all going to look like idiots and embarrass yourselves, and you'll have to get over it. Best you do it here now rather than in front of coworkers in a few years. She gave us skills and techniques that I still rely on today, and most importantly, the opportunity to practice.
I was a introverted going in to college, and that class didn't turn my into an extroverted social butterfly in 4 months. However, I have never been paralyzed with fear when required to talk in front of a crowd, and that has been invaluable in my career.
I had an English teacher in high school where everyone had to alternately do a 5-10 minute talk on a topic of their chosing at the beginning of class. They were rarely much good but it was a good exercise.
If kids were more aware how much public speaking is tied into their social and especially romantic life later they’d all be majoring in public speaking :)
If kids were aware of how high they can get in a company with talk vs work, they would think all adults were morons. So your approach has an important double duty.
Yes, yes, yes. Thank you for posting this. It drives me absolutely freaking crazy when people do the "Oh, I'm just not _____," having essentially decided they are not _____, and therefore will never learn or change.
Like every time I hear "Oh I'm just not a math person." It's not that I expect everyone to like math because I do, not at all. But I do expect people to not artificially limit themselves. :/
I have always disliked public speaking and oral defense since I was a child, to the point of fairly profound anxiety. But please withhold you woe—these people are, as I am today, adults. Masters students. Coddling them and holding low expectations has proven to be a sort of “killing them with kindness” approach to handling modern youth. People need to be encouraged (and required) to do things they don’t like to become proper functioning adults. I didnt like it, and I still don’t, as it the case with most everyone. But we all grow from it.
"Woe" is also sometimes used to tell someone to just suck it up and deal with it. In those cases it is a way to make fun of someone for being too sensitive or complaining too much.
sure, but what I was pointing out here is that "woe betide you" is a specific idiomatic phrase. it's usually used as a threat, i.e. "woe betide you if you do that".
I didn't see it as a threat here at all. I think it was just pointing out an underserved part of the population when it comes to this. And I think "withhold your woe" was meant to say that getting out of your comfort zone can be beneficial sometimes (or in other words, that it's not necessarily actually underserving).
I think we're talking past each other here :) I wasn't speaking of whether it was a good or a bad thing, I was just pointing out that "woe" meaning "a misfortune or calamity" is the only meaning of the word that applies to the phrase "woe betide", the more common meaning of woe meaning grief or sorrow doesn't apply here because it's an idiom, not two separate words.
I was pointing it out because the first comment I was replying to was clearly unpacking the word "woe" from the phrase "woe betide" and I figured they would appreciate the linguistic footnote.
I used to throw up and faint as a youngster, when having to do talks. Was unable to call people on the phone for years. Anxiety is a bitch.
However, the only way out is through. I confidently speak in front of 100’s regularly. I can pick up the damn phone (still a twinge of anxiety, but nothing like it was). The cure that works just seems to be “do it until it’s mundane”.
As an aside, it’s vital from the career perspective. You can’t let fear, no matter how extreme it feels, stop you. It’s a terrible cliche, but still true “no courage without fear”.
If you aren't capable of this, in most careers you won't be able to use your degree anyway. The only other even marginally acceptable compromise is something like the essay portions of what used to be sit down tests with only pen and paper, but longform and supervised closely.
(Note I say this because optimally, this encourages less "supervisory" desires by others later in one's career. If your peers and overseers get out of your way, maybe you will relax more and not feel so anxious all the time. Sorry for the second comment not an edit, testing new HN clients.)
Classic “get over it” condescension. Some people are unable to control their strong physiological reaction to in-person judgement. Go tell professional orchestra players who rely on propranolol to control their stage fright to just grow up and start becoming a “proper functioning adult.”
I have approx 10 meetings a week, usually more, sometimes less. Let's say around 8 hours a week spent in meetings, sometimes with people who hold the power to fire me.
Depending on your role this number can be much higher.
I'd say "get over it", because this is simply a fact of life and you'll have to talk to and present in front of a group of people at some point, perhaps regularly too.
Good luck working in the real world if you can't handle interpersonal interactions or being judged. The coddling has to stop. It's doing no one any good.
Welcome to the adult world. When I get worked up from a interaction, by BP goes into the "holy shit you need to be in the emergency room right now before your heart explodes". And not metaphorically, I have been sent to the emergency room and admitted about a half dozen times for this.
I take my meds so that my heart does not explode from talking to a person. So yeah at some point, deal with it.
Clarification for non-Danes: this post is not entirely accurate. Not all Master's courses in Denmark have oral examinations, and many courses before the Master's level (including elementary school) have oral exams. It is unrelated to the academic level, and instead related to the academic discipline and the size of the course. The latter is in my experience the main constraint, as oral exams are difficult to scale. They usually take 20-30 minutes per student, and if you have a course with 200 students, well, you're in for a long time.
The oral exam format described is also somewhat unusual, in my experience. It is usually the case that there is one internal examiner and one external examiner in the room. More than that would be ruinously expensive. The format varies; usually it's a combination of a short presentation and questions asked by the examiner. Drawing a random topic is very common.
Source: I am a university teacher in Denmark who teaches master-level courses without oral exams, and I have been an examiner for many oral exams at university level.
64 year old nerd with social phobias here. When I was in school, I had to practice oral knowledge tests since grade one. I had my first oral exam in grade six (math) and 2-3 oral exams every year since then. If I hadn't had that experience, I would never have gotten into university. I still don't like verbal communications with people (especially in a tense situations like exam), I'm slow etc, but it's manageable because I had to practice it a lot. I don't think I would have survived in modern "the environment must be made suitable for everyone" school/university.
I’m a generation younger but I concur. I hated presenting and my anxiety was crippling. But the solution for that is to practice more, not to avoid it.
100% and I think this is echoed across the thread.
It does apply to many many things that are uncomfortable/hard.
You somehow have to power through the part where you suck a lot and don’t worry about it.
Mastering that is a huge advantage in life
I have some anxiety about public speaking or presenting to a group. Not debilitating but more than normal, or so I feel. But I find that once I get going it fades. Starting off is the peak of stress. So if the examiners are trained how to encourage a person to start talking, perhaps with some introductory conversation, and when questioning not being confrontational, that might help. The goal is to find out what they know, not how polished their presentation or debating skills are.
Would it help to be interviewed in a room where there is no other person, and you can be seen through a camera and communicate through magnetic speakers?
> Of course, woe betide those with social phobias...
The professors at my grad school had a very amazing tact to get through this as well. They had enough time for specific students who they knew struggled with such issues during the mid semester orals and then had an amazing way of getting answers from them even in the discomfort.
It is amazing how much can be done with soft talking, slow progression and pauses in between from professors who care.
>>> Of course, woe betide those with social phobias....
True but maybe a blessing in disguise. As a shy college kid once (although not with any real disability), being forced to get up in front of people helped me get over it. And I was 10x better after a semester of teaching during a break in my career.
So, I grew up in the former Soviet Union. Starting with ninth grade entrance entrance and exit exams were like that. I remember I had to take university physics exam just to be able to qualify for a consideration. It was fairly rough, but I think in a way, good system because it sifted out the wannabes early on.
Yes, I did Physics, Geometry infront of the committee to explain my answer. Language course was written under a dictation, one mistake would cost you a point out of 5. So you had to be flawless on everything, spelling, punctuation, structure, etc. I aced my physics and language, but got a C (3) in geometry, I fumbled on one question.
Same. The focus on written exams rated blindly and uniformly at the US higher education system always puzzled me. I guess that's the only way to scale things up when you pay professors and teachers living wages?
the STEM academic world in the Soviet Union was extremely meritocratic. largely shielded from outside influence and maybe the best talent pipeline anyone has ever invented to produce scientific expertise. Exception being biology as Lysenkoism at some point infected the field.
Corruption affected history or economics because it was ideologically relevant to authorities but math and physics proceeded on its own.
> We ask Master's students to give US instruction on a topic (drawn randomly on a card in front of them, from a list they know a few days in advance)
This is also an absolutely great way to ensure the participants really understand what they’re saying, even aside from ai-related concerns. You could always pull definitions straight from textbooks, and write something which mostly makes sense but that you don’t really understand. But you really can’t teach something unless you do understand it, imo. The best command I ever got on a topic was due to TA’ing
This happened to me back in the 1990s when I was assigned presenting Boltzmann machine to the class. I read up on it naturally, could explain each step in isolation but couldn't still quite internalize how it all came together. But as I was presenting it, all suddenly clicked into place. It was remarkable transition from loosely connected foggy concepts to instant clarity which made me break mid sentence. Once re-composed I couldn't help but notice that the manner of my presentation has changed.
The professor (strict uncompromising man hated by most of the class) then remarked, "now I see you really figured it out".
This is not general to all subjects in Danish universities. I studied physics in Denmark and while the majority of my exams after the second year match this description (though not all), my impression is that this is not at all the norm. Students in the humanities in my dorm, for example, seemed to have an overwhelming majority of several-day take-home written exams (with no associated oral defense).
I expect a lot of the social anxiety can be mitigated by arranging things so that students get a chance to know the examiners and build a rapport with them ahead of time. (E.g., if they’re an instructor then make sure they have office hours and encourage students to go.) I’ve got my mental health challenges but that’s how things worked in my grad school and I got by just fine.
> I expect a lot of the social anxiety can be mitigated by arranging things so that students get a chance to know the examiners and build a rapport with them ahead of time.
That's not my experience. Standing in front of somebody who's been your teacher and you might look up to is way scarier than some rando.
Fairly normal, in my experience, for Masters with a thesis in the US as well. And I have not uncommon group project presentations undergrad as well--in addition to grad school.
As an ADHDer, I think talks and discussion are way better than written exams. The most common treatments for ADHD are stimulants, but human interaction is a natural stimulant, and the real-time feedback of a human audience is more engaging.
Then maybe you're not suited for the task. Most intellectual professions (keyword being profession, please don't bring up the lone mad scientist fiction thing) require the ability to communicate clearly.
If you're blind, you won't be a racing driver, that's just how it is.
That's how my school did it even for high school courses. End of the year exams were infront of the committee. Same thing, you pull a ticket and have few minutes to prepare the answer. I aced physics course, that was a proud moment but credits didnt transfer when we moved overseas...
This is also the case for many graduate programs in the US as well, but what this article is suggesting is that there are now specific oral exams to defend a student's paper, not just general oral exams.
That’s not entirely true. It varies from master’s programme to master’s programme even inside the same Danish university. Assessment is defined by the corresponding programme board.
Thesis/dissertations do have to be defended, but for course assessment that is not a given.
(I run a course and teach as well on other Master programmes.)
that's interesting, i do something similar when helping my 16 year old with his homework. I ask him to teach me the topic first (it's been a while since i took physics and calc hah). By the time he's able to answer my questions so that i can help him then he's answered his own questions pretty much.
I do this at work too. When doing code review i actually do mostly listening and then some questions. If the developer can teach me the code and i understand what they're doing and why then it's usually pretty good code already.
I can see this being a problem in the US when it comes to "accommodations" in general. Students with an ADHD diagnosis can claim it causes just about any difficulty and get accommodations for it. The local (top ranked, but everyone's seen all the stories coming out of the Ivies...) college here gives students with ADHD double time on exams and has them take it in a special disabilities proctoring center where it's a struggle to get the staff there to even enforce those time limits. Obviously this also makes it infinitely easier to outright cheat as well.
Not a chance you will get past the "my ADHD makes it hard for me to think about these things verbally" challenge at institutions like this.
Oh, at least in the post Soviet tradition of oral exams, you are given plenty of time to think and prepare your argument before you start an actual conversation with the examining professor. And if you need to think more during the exam to answer a tricky question, the professor just shifts to talk to another student who is ready to present their case for the time being. So it is less of a public speaking and more of a conversation where you can take a pause to think through an argument, or a proof, or a problem.
as someone with ADHD who always finished exams first or was the designated spokesperson for group presentations...
I have no idea why you'd want to be taking a test a second longer than you already have to.
Either you know the material or you don't, anything taking a lot of time is borderline stuff where you're scrapping for points you might not deserve in the first place lol
> I have no idea why you'd want to be taking a test a second longer than you already have to.
The theory was that ADHD students have trouble sustaining concentration, so extra time allows for thought breaks and distractions. The idea was that 2 hours for the ADHD student would be similar to 1 hour of concentration for a non-ADHD student, and therefore the inequity had been partially compensated.
The teachers I know have the same observation that you do: The extra test time isn’t usually used by the ADHD students, but when other students (or maybe more accurately, parents) discover that extra test time was available, most of the high performing students suddenly got diagnosed with ADHD at the same time. They were the ones staying extra long to proofread their work.
For the elementary/junior high teachers I know the schools’ solution was to give everyone the same extended test taking time. Easiest way to remove the incentive to seek out a fake diagnosis without taking anything away from anyone.
Absolutely not saying this is even the most common case but: people “without ADHD” but with a diagnosis can use the extra time effectively.
It is most clear when there is some kind of problem solving involved. An extra 30 min to figure out the trick or plan a detailed approach is a real benefit.
The other case is where there’s some missing knowledge but with a bunch of extra effort you can figure out how to do it. That would be something like re-constructing a self-balancing binary tree from some half-remembered lecture notes.
ADHD is known to be able to affect speed and consistency of output, even of mastered material, and in a variety of ways -- hence "I have no idea..."
Standard diagnostic questionnaires commonly reference this. The last part of your comment reflects a lack of understanding of both human cognition and recall and ADHD in particular.
Treating many ADHD adult students some did take advantage of accommodations available to them.
Problems with test-taking varied, but common issues were getting distracted by extraneous stimuli in "regular" classrooms. Frequently ADHD students were slow readers and really did need/use extra time. But the biggest problem was test-taking anxiety, often quite prominent, contributing to difficulty with tests.
Interestingly, more than not ADHD students were reluctant to ask for accommodations. Some never availed themselves of this benefit. Others only used part of the options they could use.
In my practice were a number of ADHD adults admitted to nursing school, some having serious academic difficulties and eventually asking for accommodations. The assistance was effective. Notably, none of these student-patients failed or dropped out, all graduated. A pleasant footnote to add is a subset of these nurses developed exceptional skill earning them leadership positions in their institutions.
Reputable clinicians wouldn't give an ADHD diagnosis or support academic accommodations without sufficient clinical evidence.
Part of the problem here is that this college has <5% acceptance rate. Very hard to get into and prestigious. The students who get in have been minmaxing and exploiting every advantage they can for their entire life.
> The students who get in have been minmaxing and exploiting every advantage they can for their entire life.
Not a school I'd be tempted to apply to. Scholastic competition is one thing, but scrapping for every conceivable speck of "I'm better than the next guy" is not a bit attractive. I wonder about the campus atmosphere. Seems likely contesting would be a hard to break habit. I feel tense just thinking about it :/
Of course, ADHD students are less likely to achieve near-perfect grades than non-ADHD peers. Though it's not impossible for an ADHD student to succeed. Anyway, not at all uncommon for a student to do very well until subjects become more complex and challenging. Conceivably a student could look promising, but then suddenly have great difficulty keeping up.
So a safe bet that at least a few students at the top-level school have ADHD. If the school offers little or no accommodation, affected students probably have to accept getting educated elsewhere. Fortunately, more than not ADHD treatment is effective, a favorable response can make a pivotal difference in the student's outcome.
> college here gives students with ADHD double time on exams
No offence, but this is a ridiculous thing to get worked up about. Various neurological conditions exhibit in people just being “slow” in exams. No one ever diagnosed me with anything (probably too long ago), but teachers somehow understood that I was one of these and gave me extra time on exams.
I didn’t want it (who wants to be in an exam a minute longer than required!). But I did need it through high school. In university I was able to complete in time (mostly); I suspect because I had practiced doing exams so much by then that I’d figured out how to optimise for time (maybe everyone knows this and I was just very slow to learn it?).
I’m someone who excelled academically, and it was an important part of my identity at that time. And I do remember exams (before teachers gave me more time) where I’d get a lower grade as I’d only completed 70%. It was incredibly painful; I knew the material, if only I could get it down on paper in time! (I’ve also been an incredibly slow writer). Extra time for someone who struggles to complete on time isn’t cheating - it’s allowing people to demonstrate what they can do.
European universities also have accommodations and extra time etc. And yes it's an arms race. Though at some point it will simply inflate away the value of the degree, but educators will stand there baffled, not understanding how it could happen.
This is how higher education was done for literal centuries before moving to the written word. So it’s not exactly innovative to return to oral arguments.
But what it does threaten to do is effectively abandon all the efficiencies of the written word.
When higher education evolved into a mass system in the 1800s and into the 1900s, pure oral examination was looked at as super inefficient. A written paper could be handed in and graded without lining everyone up in front of a panel.
The oral system scales poorly. One examiner can only hear so many students in a day. There are damn good reasons Medieval universities were small and modern ones are not.
Requiring every major paper to be defended live reintroduces the bottleneck that written exams were designed to escape.
Accessibility takes a hit too. Students with speech differences, anxiety disorders, hearing impairments, or simply less fluency in the language of instruction face an extra barrier that a written submission does not impose. The written form, for all its flaws, leveled some of that ground. Shall we abandon the differently abled because machines disrupt the academic system and essentially make cheating effective? This does not seem very logical, or sensitive.
So while it’s nice that academics are rediscovering the old ways, I don’t really see this as a scalable answer to raising education levels for the masses, but more of a knee jerk way to try to bypass the obvious technological disruption by returning to non-scalable, inaccessible ways of doing things that our ancestors abandoned for a reason.
It has its place for graduate students and smaller volume education systems. It’s fine for keeping the ivory tower clean. But it’s not the answer to expanding education to everyone and taking advantage of the technological disruption to advance education and learning throughout greater humanity, which is what AI definitely possesses as a possibility.
Oral exams are still common in Germany and many European countries in case of smaller courses taken by 20-30 students in a semester, and sometimes even in larger courses, and the task of examining is then split among multiple professors and lecturers.
In Hungary it's a pipeline. You go in a room, you pick a piece of paper out of a hat, which gives you some task or topic to prepare for. Then you sit down at a desk (the room is filled with about 15 students working already), then you solve the problem or sketch out your notes to present the topic you picked for like 20 minutes, then you go to the professor and you have to, in low voice, explain your solution, the professor asks back and pokes at it to make sure you understand. The professor then gives a grade immediately and you go home.
In Germany, it's usually no random pick of a topic, you just go through the entire semester's material over 30-45 minutes and the professor stops to spend more or less time on each topic to explore your knowledge. You can explain things by making sketches on paper. A PhD student sits there quietly taking notes of what was asked and what the student answers approximately. You leave the room, the prof and the PhD student discuss the performance, decide the grade and you come back, and the professor explains the grade and what you did well and not well.
I'm not familiar with the German universiry system, but I think a 1 hour, 2 professors (or one professor and one PhD student) would be impossible to scale. A class of about 100 students could take an entire month! (or better, 2-months of professor-hours!)
One big difference I observed between the US and $HOMECOUNTRY is that US colleges require a lot of courses with little credit, so a normal college year is N courses x K credits per courses, N large, K small. In $HOMECOUNTRY it is typically N small, K large - think instead of having Calc IV, Trig whatever, Algebra 2 and Introduction to Proofs, you have a single "Mathematics" course that last a whole year, not just a semester.
Since each course needs its eval, assessment is a function of N, not N*K or K. Much easier to scale if there is ~5 evals per student per year than ~20.
But that is how certain exams are done right now. Though it's usually 30 minutes including overhead. It's not something you can do in classes with multiple hundred students, but it's certainly done at scale.
I was the PhD student in the room in dozens of these exams, and I toke some of them myself. But that was for classes with a bit less than 40 to maybe 60-80 students. The load for the larger ones was shared among professors, if I remember correctly. But it was certainly the case that a single professor spent 1-2 days doing nothing else but these exams.
And it was 1 professor and 1 PhD students, and those don't count. They essentially grow on trees and are always available, if you need some.
I think education is something better scaled horizontally than vertically. More schools, with more professors instead increasing the student per professor throughput. The classes where I learned the absolute most, and had the best experience, were closer in size to 25 than 100.
The US has proved it can do horizontal scaling, as the majority of adults gets a college degree.
The horizontal capacity is ineffectively used by having many degree mills and low-value degrees, but if we re-tooled the knowledge sets we could raise the bar.
At low student counts, like 20, (think specialist niche elective master's courses, not the huge bachelor's courses) it's perceived by profs as more efficient in fact than the ordeal of organizing a written exam.
With an oral exam, the prof blocks off 3 days or so, with ~8 students per day and it's done. It's not a huge additional effort compared to all the effort of giving the course over the full semester anyway.
An added, though minor benefit is that the professor might decide to offer a student assistant job to someone who performs well, because it's much easier to judge who is actually good in an oral exam vs in written. There's a reason that job interviews aren't just tests filled out on paper but involve personal interactions.
Oral exams can also compensate a lot for different types of exam anxiety. Blanking on individual points doesn't matter as much, the examiner will help you along or ask along a different path. Doesn't help for someone that completely shuts down, but it's much easier to determine if a student was just stuck on some minor point and actually knows the topic in an oral exam.
They are likely prone to more biases than written exams, and the variability among examiners is likely high. My experience is both in taking these exams myself, and also sitting in on them as the note taker many times. Though that was only with one specific professor, and almost always on the same course. But that was in my experience a very fair exam, even for those students that were very noticeably nervous.
I think you have the right observation but the wrong conclusion.
The appeal of digital systems, at a social level, is that they can transmit information accurately for almost no marginal cost. They come at the risk of fraud.
With AI the levels of risk have increased. Anyone can now be a fraudster, and it gets harder and harder to prove anything online. Either of us two, in this conversation, could be a well tuned LLM.
So society, collectively and at an individual level, is forced to reassess reliance on purely digital information. Which means that the advances in the technology, actually end up reducing the utility of digital systems more generally.
You can argue that this makes education costly and less accessible. It does. It absolutely does. There is also no alternative except to see higher education get wiped out, which would have devastating economic and social effects.
"So we are going back to blue books..." exactly. And, it is also efficient. You just need a proctor to ID the test taker and make sure they aren't using AI to cheat in order to verify that the student learned something which seems like a good idea.
Even in fairly recent times viva voce was common. I had to defend my final year report on a long piece of practical work as part of the final for my BSc in Applied Physics in 1975.
You don't need to do everything orally. A return to handwritten final exams done without electronic aids would help too. They can be open note as mine were in the mid 70s or even open book.
As for scalability, I really don't see the problem; if we are producing so many qualified people then we surely can use some of them to examine the next generation. It sounds like the argument that often appears on the web where someone, usually an American, claims that a solution that works in, say the UK, can't work in their country simply because their population is larger.
Also, we really should consider if shoe horning every young person into an academic institution is useful either to society or to the person concerned. It certainly has not been an unmixed success in the UK.
I graduated in 2000 and remember doing the oral defence. It wasn't a huge deal. You already have a personal relationship with your tutors and they are familiar with you and your communication style. They are not the Gestapo.
This is basically my thought- my school is considering all sorts of things to combat AI cheating, and most of them scale poorly. You can implement them and increase your reputation, but your tuition will have to increase to pay for a lower student/teacher ratio.
So we’re probably going to see a split- mass production schools that still act as job training for Industrial Revolution era jobs will keep doing what they are doing. Schools that want to keep their academic prestige will implement these inefficient safeguards and raise prices accordingly. Employers will distinguish between these two types of diplomas when there are enough options out there that are combatting cheating- much like in-person schools are generally valued higher than online educations, or how certain schools (MIT, Stanford) are valued higher then others. This might lead us to the transformation of higher education many have been anticipating, because the current system cannot remain as-is. We all thought technology was the thing causing the upheaval but AI has really accelerated things.
And what a better time to set optimization aside when schools and universities are getting less and less students each semester due to population decline.
Interestingly, you dont consider your 'inverse' point: that now the manual part of teaching the stuff can (and definitely will) be automated with LLMs.
It will become similar to coding: humans evaluate, LLMs just provide what has been mechanical for a couple hundred years
It’s a good answer for keeping online education honest. It doesn’t need to be applied to all in-person schools, where teachers distribute spot checks across the course relatively evenly, but it makes perfect sense in conjunction with online testing. You only have to challenge one essay per student per term to catch cheaters with a near-perfect success rate. Assuming a 12 week course with 36 students, that’s qty.3-4 of teacher:student one on one meetings per week for the term; I think that’s doable. If you’ve oversold your teachers and have more than 36 students per, guess you’ll have to increase your labor expenses :D
It shouldn't take long for educators suss out whether students know the subject they are talking about. You don't have to question them exhaustively to determine that they likely did or didn't do the work they are claiming. TFA is about high school projects, not PhD defenses.
Universities in the US charge fortunes and pay instructors a pittance while plowing ever more money into “administrators”. Fire half the administrators and hire more instructors to both proctor exams and give more personalized instruction.
> Students with speech differences, anxiety disorders, hearing impairments, or simply less fluency in the language of instruction face an extra barrier that a written submission does not impose. The written form, for all its flaws, leveled some of that ground. Shall we abandon the differently abled because machines disrupt the academic system and essentially make cheating effective?
Universities are entirely elective, why would they need to cater specifically to disabilities? Driving schools don't make exceptions for the blind. There are skills that people expect you to have with a certain kind of diploma, and it reflects poorly on the school if they don't have a standard to hold people to. Not being able to present your work in a basic way is one of those mandatory skills that imo makes you unqualified to graduate if you don't grasp it.
I'm not saying you can't make an exception for a person or two who is clearly as good or better than the rest otherwise, but people will always try to misuse as many exceptions as they can, en masse, as an excuse to just avoid doing required work.
I feel this was already a severe problem pre-AI. From what I hear from my partner (in university administration), nearly half the students demand accomodations for disabilities. Many of those conditions are very difficult to falsify
It doesn't feel good, but at some point we have to ask, can we support every single case, at the expense of the whole? I'd like to be convinced that's not a necessary tradeoff
I don't think it's that bad yet, but it's certainly on its way there. We're like at this intersection of lots of conditions that previously weren't documented becoming actual diagnoses, that pulling along a lot of peer-pressure kind of trendiness to have some kind of fault that makes you more unique, and the practicality of a system that was designed to give massive benefits to rare exceptions, for which now so many easily qualify on paper.
The list of conditions that qualify for what will most likely have to be redone completely from scratch.
I don't think the world really works like that. Hearing an elevator pitch has a very different impact from reading a short-form business proposal. And of course the efficiency of the written word is not absolute; in lively discussions and debates, there is no time to put thoughts down into the written word; they had to be uttered. I am personally of the opinion that a successful university graduate should not only be able to write down their ideas in words but also be able to speak to defend their ideas.
"The oral system scales poorly. One examiner can only hear so many students in a day."
I'm not joking: you could have an AI quiz the students and test their understanding. That way the tests go deeper into testing the understanding by following through on the answers, and are standardized and fair for everyone. I don't think we are there yet at all but I actually think its not insane for the future. I expect downvotes though/I don't think our generation would agree with this.
But even with the written exam, wouldn't the bottleneck still be there where the professors still need to read through all the papers? I guess its definitely more efficient and asynchronous, but I can't imagine by much assuming the same and equal rigor is applied.
As an educator myself, this quarter for the first time for my course's final project, I've switched to actually asking my students to provide an "AI Authenticity Audit" of their (project-relevant) chats in the construction of said projects. All my students' output over the last two years went from pre-AI level quality to boardroom-ready polish, and I'm actually more interested to see how they arrive at an outcome rather than the final output itself - focusing on their questioning, curating, editing, and iterating process. Didn't think we'd get to somewhere like this so soon, but here we are. Strange, but I'm trying it out.
IMO this kind of thing adds work and stress for the best students. If I'd been faced with something like this while I was in school I would have felt the need to basically curate a parallel construction AI discussion, knowing it would be judged. I understand the theory that, in the age of AI, the prompts are more interesting than the outputs, but I'm not convinced this is the way.
Things like the Denmark approach (and/or in-person written responses) seem like a better direction. Treat take-home writing the same as we've treated take-home math questions - as useful practice (for those who choose to actually engage with it), but not a significant input to grading.
Yes, I thought about that, and which is why I'm just running this as a pilot for this quarter to see how it goes and I'm not particularly attached to it if feedback/results are mixed. But that said, there are real constraints on my ability to supervise students with in-person examination, but that is more a problem with how my society funds and cares about education. We are forced to take approaches that 'work at scale', hence experiments like this. But truly, reception was positive when this was introduced, because students too want to be held to a good standard and don't want others to get credit for not doing the work toward achieving the learning outcomes like the most honest ones.
Oral Defences have a long established tradition in Denmark, but recently there’s been real cut back on this as a money saving exercise. So both teachers and Students are familiar with this form of examination.
So for a dane this reads like “back to the old way”, rather than anything new or novel.
I'm quite impressed with the Hungarian system, for 18-19 year old school leavers, where every subject is split 50/50 between:
- Írásbeli (written) - standardised written exam, centrally set and marked
- Szóbeli (oral) - the panel interview
The oral (szóbeli) has:
- A committee of 2-3 teachers sits as a panel
- The student draws a topic at random from a list
- They get prep time (usually 15-20 mins)
- They present their answer to the panel
- The panel asks follow-up questions and probes understanding
In my high school English class (like 10 years ago) my teacher assigned readings and called on a random person each day to give an overview of a section. He basically quizzed you in front of the class on themes, plots, etc. It was very effective at getting me to read and care for that class, because if you just sat up there and didn't know anything you'd lose marks on participation (and look kind of dumb). Not sure if/when/why oral presentations and public speaking are no longer got prioritized.
I had to do an oral defense of my thesis in graduate school. When you've spent months and months writing something extremely obscure... well, I'd hope that it's not that difficult.
A fun anecdote from my defense was that this element of logic "did not exist before some time" or something.
One of my citations was a book from like, the 1700's or something, and it absolutely blew away one of the committee members that I'd found it. He asked me "how on earth did you find this reference?" and I just told him that I look at all the math/logic libraries in Boston I had access to, I looked up all their logic books that predated the cut off I was looking for, and I skimmed them.
I didn't even have to leave my university's library, because one of the first five books had the counter-factual I was looking for. Sure, it was a rare book and I had read it in a special room, but it wasn't particularly hard to find.
Before that defense, I was 100% the only person alive with that information.
The problem with oral exam is that it’s done serially student by student, which can be infeasible with large classes. The grading is also highly subjective.
Written exams are also not best suited to project-based classes.
What’s the best way to grade students in classes with coding projects?
Its serial nature also means that an evaluator's standards can drift from students with early exams to those with later exams. Oral exams can introduce issues of sex and ethnic biasing.
I'm teaching a university math course next semester and I've been messing around with a solution to this that my old analysis professor used to employ. Every week, they take home a handful of theory questions. These aren't turned in for credit. But, every week, there'll be a short quiz where one of those questions are asked verbatim, along with a small follow up.
That felt like a very natural solution to me. Of course, worst case scenario, they just blindly generate and memorize the proofs without thinking. But I think the follow ups will catch them in that case. Moreover, I choose to believe that >90% of my students are willing to at least put in the work of scrutinizing those proofs which is honestly not bad exercise in its own right.
I would love to do oral exams directly, the logistics with the number of students I have just make that hard. I know we're trialing it with the less populated special topics courses.
I wish we were doing this in the US even before AI was an issue. I suspect it would reorient students to their work in some pretty cool ways. I mean, it might feel like torture for some and maybe that’s not so much a bad thing. I’m sure I’m not thinking of some downsides to this too, but it seems like such a great counter against “factory” schooling. Of course, this could be a lot more work for faculty and could be hard to keep up with a huge student population. Again, maybe a feature more than a bug.
It was already long-standing practice, when I was teaching many many years ago, to interview suspected cheaters (it's usually quite obvious) to see if they actually knew what they claimed they knew.
Only the dumb ones are obvious. In my school (SFU/BC) most of the students in almost all of my classes cheated. VERY FEW got caught.
I heard instructors say all too often that it was obvious when somebody was cheating, but somehow they missed the guy paying his cousin to write the final.
This just feels like good educational practice too? I remember really enjoying TA'ing as an undergrad for courses I took the previous year, because THAT'S when I felt I _really_ learned the material.
Likewise, when I was a lab consultant, I felt like advising others was when the concepts of Linux/systems administration _really_ sunk in.
I always crammed the bare minimum for any kind of assessment (by the end of college I was actively avoiding any classes with final exams), but when I need to be able to teach someone else or explain my understanding in a live environment? That's when I _actually_ learn.
It feels like systems like the Maturita' final multi-day exam in Italy do a pretty good job at providing students with a standardized, state-supervised process for demonstrating one's competence by the end of high school. No computers, you're under the watchful eye of your teachers the whole time, nobody knows what's going to be on the exam until the day of.
When I was in school there was a maths aural test. The teacher read out questions and you wrote the answers, with no calculator. Basically a stupid name for a mental arithmetic test. The funny thing was the teachers were so at pains to differentiate between two words that sound basically the same in our local accent, aural and oral, that they insisted on pronouncing aural as "ow-eral' the ow being pronounced like a pain reaction sound. The smarter kids took to teasing the teacher with words like aura and aurora, and asking how to pronounce them...
Fun story: Final oral exam (in history, I believe -- a thing back then), I took a whole A4 paper and turned it into hundreds of little balls in front of 3 teachers over the duration of the oral exam.
I had no idea I was doing this until one of the teacher pointed to the heap in front of me, but it saved my bacon as I did otherwise well.
in the french CPGEs they have you make a mini research project to be presented in front of 2/3 judges at the end of the 2 years, each year has it's own theme (this year it's efficiency/optimisation)
thankfully it doesn't need to be very deep, but it has to tie somewhat into the curriculum which can be annoying
Related, I've moved to requiring regular in-class handwritten quizzes. Just a blank piece of paper to start, with the questions on the board. The practice forces students to recognize whether they are learning specific topics.
This is how we used to grade labs when I was a TA in college. Students worked in groups of two and showed up with a working project to get graded. Afterwards, we'd ask the two students questions and it was extremely obvious if one or both of them didn't work on it.
As a thought experiment:
In-person testing or oral defense approaches work until we get implants. If people have implants, what then? Demand that the implant be turned off? Demand that the implant's data connection be turned off? In the latter case, the person with more storage or a better local model wins.
In the long run, in the AGI world, if everyone has implants, then everyone is roughly equal because computer-aided computation will be better than what any individual can do. Those without implants are screwed.
(I don't want an implant; nor do I want others to have implants.)
Thos whole debate is happening because people think a degree matters. Because getting a piece of paper is more important than knowledge. Students will continue to cheat (AI or not) as long as the goal is to get a stamp.
In my uni years bachelors, masters and PhDs required thorough research resulting in a huge opus printed and stitched together but the only thing that really mattered was the presentation of the work in front of the committee and oral defenses. To this day I have been living in full confidence that this is the academic standard that has been in place and effective (with variations) over centuries, if not millenias…
Even intermediate exams were a combinations of practical work in some written form and the theoretical grill sessions in the end of every semester.
All in all, if I had to put a professor’s hat today I would not give a sh.. if my students would use LLMs to write text. What would matter (and what does for me as a hiring manager at $WORK) and a human being with decency is the results of the _natural neural network_ training so to speak. IF whatever prompts and interactions with LLMs result in sharp knowledge of the subject and ability to apply the knowledge to solving real problems, well done and congrats!
This is a great strategy. It is something I do in code reviews as a senior developer. There has been observed longer code reviews for seniors and this kind of approach keeps it manageable.
This is great. Sorry, but the Western World has to stop normalizing extreme introvertism. Humans are social. We rely on communication and cooperation to survive. Allowing kids to shelter themselves from interaction and communication during crucial development stages will only hamper them in life.
maybe the policy will succeed in encouraging learning for most students (at least I hope so!!); otherwise some might still rely on ai because of the trust they've built up with it, a lack of connection to and therefore faith in the info they're expected to just memorize, low confidence in their abilities, etc
Of course they could, in fact it'll probably be great prep if they read it and understand it instead of just memorising the answer.
If they just memorise the answer any smart teacher can trip them up by just rephrasing things or developing tangents to see understanding.
In school before exams, I'd often make little cheat sheets about things that you'd need to memorise like formulas for areas and volumes or certain physics equations (we weren't allowed any reference material for that on exams). The very act of writing them all down before the exam would often boost my memory enough to the point I'd just recall them.
I find it quite surprising that Danish high school students (the article isn't about university) apparently did write digital exams with internet access until now, so basically open book.
I mean, this is pretty much the easiest way of handling the problem.
When I was teaching in a university the easiest thing to test someone's understanding was a simple question: "Explain me how would you use this in a day-to-day situation."
If the guy knows his business, he'll go on a long lecture obsessing over every new feature of C# 2.0 or PHP 3. He would give you all the differences between Firefox 3.5.1 and IE 6 with Maxthon. (Goddamn I'm freaking old)
Should you not find a spark of interest but just random mumbo-jumbo of words that don't constitute sentences, you can clearly see that there is no understanding present in the subject.
In this case you work as a teacher, helping the guy to figure out what he does not understand in a subject and simplifying things for him till this clicks. After that he will be able to talk about said subject for hours with gusto.
Somehow Russian technical universities were never relying on written work, there is some, but even that is is delivered in person with the tute teacher asking this and that.
Tangentially, the market for uni assignments (mostly humanities) for pay was huge before AI.
In my part of Europe oral exams have been a staple even in highschool and across unis. I was pretty surprised when I've seen that other parts of the world don't complement written testing with oral understanding checks.
This is a good move. I teach a summer async remote course that's ending next week, but I'm giving all the students incompletes until they come back to campus and take an in-person exam the first week of classes. All homework is worth a marginal amount, and if they fail the final they fail the class.
I'm not going to spend my effort policing whether they're using AI or not in their class work because it's a losing battle. At the end of the day, they can either demonstrate mastery of the material at an exam or not. It makes the exams higher stakes, but that's where we are.
I hope at least the homework is good prep for the exams, otherwise if its grade value is marginal there's little point from the PoV of a student that just needs to pass the class to take it seriously
I'd never pay to have you as a teacher, I'm sorry. Maybe your move makes a sense for the reputation of your university's degree, but as a student willing to pay to learn remotely, I'd not accept to be treated like children.
Regular graded assignments is the "treated like children" version. The adult treatment is that you can come to class if you want, you can do the assignments if you want, but if not, that's fine too. There is one test at the end, and it's your responsibility to prepare for it as you want. You manage your own time, we offer consultation, lectures, assignments, tutorials etc. but you can reject all of this and learn the material from some other textbook, and as long as you've mastered the material, you'll get a good grade at the exam. This is how German universities work, for the most part.
The problem is the vicious feedback nowadays of online student evaluations of professors. Professors that are rigorous get complaints and low rankings, kids don't sign up for their sections etc. Kids (parents) pay the administration and the administration wants to keep the customer happy. The only real answer are standardized exams (such as licensure exams) which are beyond parent pocketbook capture.
Or like Europe. You don't pay tuition, but if you slack off, and don't progress properly, you only have a limited number of exam retries and limited number of funded semesters. If you fail, you're out, choose another profession. You're not a customer, you're someone who is trying to prove themselves and get certified by experts on their knowledge. The "customer" of the university is whoever will glance at the degree paper and tries to ascertain the skill/knowledge level of the outgoing student. They have to give a good signal there. The reason this doesn't work in the US, is because they want mediocre children of rich parents to be pushed through, because college is a networking and social class-based finishing school, not a place for acquiring knowledge.
(Also, there should be electives, but the rigorous backbone of a degree should be mandatory, so "students won't choose this course" doesn't come up.)
> The reason this doesn't work in the US, is because they want mediocre children of rich parents to be pushed through, because college is a networking and social class-based finishing school, not a place for acquiring knowledge.
Out of curiosity, which US university did you attend? I went to a well-known and highly-regarded public university in California, and they absolutely did not coddle their students. Poor-performing students regularly washed out.
None, but have interacted with American students at European universities and they were very lost without all the coddling and all kinds of support infrastructure and basically getting their life organized by administrators who personally attended to their matters and reminded them a million times about deadlines and so on. They were unprepared for actually having to manage their own studies, read up on rules, find out where to register for something, until when, etc. etc.
That happens when American students first join US universities too. It takes a few weeks for freshmen to get acquainted with the new reality of being completely responsible for handling their affairs and ensuring their success. It’s nothing to slam them about; that’s an artifact of coming from a system that provides more guide rails during primary education and childhood than European children might enjoy.
American colleges give way more support throughout the studies though. In Europe you're literally alone and nobody cares if you miss deadlines or you're failing out. No general consultations, no extra departments and admin bloat. You have to have your shit together.
It really depends on the university. Mine wasn't particularly helpful in this regard. They sent me a form-letter warning by mail once, when my GPA sank too low one quarter (this was before email was ubiquitous). And they did have some counseling resources available, but the resource-to-student ratio was very, very low. I would not call it white-glove service by any means.
Perhaps you also don't have a comprehensive picture as to what is available to students across every university in Europe. Europe, like America, is a big place and, like America, the situation probably varies by institution. As with most things, the answer is probably more nuanced than general observations would make it seem.
Tenure protects professors against students' complaints (often for the better, sometimes for the worse). Also, the administration already knows that students complain about professors who are tough, and don't typically pay heed to those complaints. Parents don't really have recourse either: what are they going to do, send their kid to Podunk State instead?
Yes indeed. College departments are well-known for cutting the ranks of the professors who have proven track records of boosting enrollment in favor of awarding tenure to the professors who can't get butts in seats.
I am very well aware, having close relationships with multiple faculty professors in my family to this day. Practically none of it has anything to do with student ratings.
Which university did you attend? I asked you once already; put up or shut up.
Why does which college I attended have more bearing than the ones I have and do work at? I don't particularly care for your stupid irrelevant demand and I am refusing to engage it.
I, my parents, my in-laws and my siblings are all academics. I don't need to prove anything to you. My brother is faculty at an ivy and literally what I am describing is what is currently going on in his department because the administration has foisted MBA brainrot "performance metrics" that are easily gamed by students and faculty who don't particularly care to teach but need points in their portfolios.
Students are entitled "I'm the customer, and I'm always right and I'm paying you a lot" twats who get the administration to bail them out if they don't get As for half-assed work. It's similar but not as bad at R1 state universities where I work and have worked.
You are completely out of touch and which university you or I attended in the 90s (two generations ago) is completely irrelevant.
Is your brother actually a full tenured professor? Does he bring in grants and endowments and is well known in the research community? Is he being threatened with serious punitive action for not giving into individual student demands? There is a tremendous difference in treatment between those who are fully tenured and those who aren’t. Lecturers and adjunct professors are frequently treated differently, and I accounted for that in my original reply by referencing the tenure track.
Students can ask for whatever they want. It doesn’t mean they will get it, unless maybe they are VIPs at Ivy League schools, and those are a sliver of the total population of university students. Don’t mistake the minuscule minority for the majority.
You’re letting your personal experience cloud the full and diverse picture of reality. Consider that we may both be right depending on the circumstances.
I don't give a shit what you "accounted for", you're hot air with no teeth in the game passing yourself off as some sort of "expert" or knowledgeable about anything. I might as well talk to a LLM.
If you only care about the learning, then you could just take the incomplete. You don’t somehow lose the knowledge you gained.
The only cost of an incomplete is it doesn’t help towards a degree, but a degree is just a signaling mechanism. If you want to be able to use their degree to signal something to someone, you will have to jump through their hoops.
As AI eats up the need for half-assed "I googled a few things and wrote something up and then slacked off for five hours" white-collar-paperwork jobs, it's gonna be pretty hard for people not willing or able to demonstrate that they can do anything that couldn't just be fully done with an "agent" instead.
What is your expectation from such a class? What are you hoping to get out of it?
If you merely want a tutor that will help you out navigate a topic you are interested in, with no strings attached, then sure, yes. No need to take exams.
But if the goal is to get a formally recognized degree in some topic, then you will need to accept that there will be oversight over how you understand that topic. Because the institutions granting those degrees need to sign them off. And if they care about their reputation, they would be interested in signing off only on those students, who actually understand the topic.
So there will need to be exams. In the AI era, it is close to impossible to guarantee that a remote exam will accurately evaluate your understanding of a topic. So the exams would need to be carried out in person.
There is nothing children-specific in using such approach. It is merely one of the few still somewhat objective ways to actually test people's understanding of some topic.
You can learn remotely in his class. That's the whole point he is trying to make. Hybrid does not mean fully remote. If you want fully remote take a full remote class.
"The student is the customer" is a stupid metaphor that only caught on because a lot of people would rather just bring in dollars than worry about "are we actually teaching anything?"
Denmark: "You must now verbally defend your essay." Students: sweating nervously "Uh, well, you see... the AI wrote it, but I agree with what it said?"
The more I read about articles like these, I grow even more convinced that the education industry as a whole, regardless of the country or stature, has kind of lost their plot.
Sometimes I even wonder if this is the outcome of sheer laziness, fear or both.
When I speak to professors, teachers and students - It's also surprising that "top tier" institutions are fighting AI harder (outside extremely specific courses like Harvard’s flagship CS50, MBA courses at Wharton (UPenn) and MIT) while "bottom tier" institutions are completely embracing it and rebuilding their curriculum around it. One CS professor at a "bottom tier" CSU mentioned to me that he's actively going completely "open book using AI" - students are allowed to use anything they want from Claude to Codex to OpenCode to finish assignments but the assignments have now changed from "blurt out quicksort" to "let's sort N natural numbers in a cache efficient way using least amount of resources". I would hire the latter over the former anytime. I am tired of interviewing candidates who can shit out quicksort before I can even finish my sentence but stare at me dumbfounded when I ask them to sort people's name serialized in unicode.
In my opinion, the bulk of traditional education has been a mix of memorization of facts, knowing inference rules, and applying inference rules to those facts, coupled with recall.
Before the age of LLMs, only very expensive-to-build rule-based expert systems were able to replicate that functionality. Humans were just simply cheaper and way more reliable.
In 2026, Frontier models are exceedingly, across the board, across industries, breaking records and challenging those notions on cost, capability and sophistication.
Trying to replicate how education used to operate pre-frontier LLMs is like forcing people to farm by hand in the age of automated tractors that have LIDAR, Vision, RTK on board.
I'm not discounting other elements of learning like collaborative debate, clinical/lab work, Socratic reasoning, emotional intelligence and the development of a professional network - but I argue these skills are not limited to a school or university setting. Infact, a lot of this is distorted in a school or university setting compared to the real world.
I have been filing my own taxes, including complexities like equity, real estate and business income for over a decade now, so it's not just a simple 1040 and 540. Reading through IRS documentation, talking to EAs and CPAs to fill in ambiguities and gaps was pretty expensive in terms of time and money.
Both Claude Opus and GPT 5.5 now, as of 2026, answer all my tax questions correctly. Those thousands of dollars in time and money I had spent has been replaced by a single $20 subscription. Unless tax codes drastically change every year, that $20 is a one time cost.
If I were to begin my tax journey in 2026, I would never had to spend those dollars: dozens of tax professionals are out of a job - all their education is for nothing.
It's entirely irrelevant whether they passed their exams by carving answers out on granite, taking their exams in a jail cell on an island proctored by Catholic Nuns under the watchful guard of automatic machine guns manned by T-1000s, I just don't need them anymore. For my usecase, these humans and their credentials provide 0 additional value over a one-time $20 expense, no matter how complicated it was for them to get their credential and what complex interpretive dance they had to do to impress the people awarding their grades.
All this "reject AI, do it by hand" is just insanity at worst and laziness at best.
That said, I'm personally unsure what the future of education in the age of tractors is like but removing weeds, planting seeds, watering them, all by hand is certainly not it.
It's my understanding that most countries - developed or developing - are bottlenecked on educating their masses due to lack of qualified teachers.
That's a bandwidth problem. Having those teachers listen to oral arguments from a handful of students is not solving the core bandwidth problem. All these shenanigans is doing a disservice to the public.
> In my opinion, the bulk of traditional education has been a mix of memorization of facts, knowing inference rules, and applying inference rules to those facts, coupled with recall.
These are important skills to develop, even in a world with LLMs.
> All this "reject AI, do it by hand" is just insanity at worst and laziness at best.
If a 3rd grader complained about having to learn multiplication because calculators exist, would you agree with them?
Why do we make kids learn how to do math if they can pull out their phone and open the calculator app? Because learning how to learn is important and there is value in understanding how the answer is produced, even if you have a machine that can produce it for you.
Universities can teach a separate class on how to use LLMs. The existing classes should be focused on teaching their existing subject matter, not becoming an extension of a how-to-LLM class
> Why do we make kids learn how to do math if they can pull out their phone and open the calculator app? Because learning how to learn is important and there is value in understanding how the answer is produced, even if you have a machine that can produce it for you.
I actually appreciate this question very much because this question is extremely pertinent to our discussion. Before I continue my response, I want to turn around and ask you:
1. What's your definition of "learn how to do math"?
i. Would it be sufficient if they proved they understood what addition, subtraction, multiplication and division was? Or do they need to be able to correctly calculate what 4592 * 314 is? What if they could show you using diagrams of squares and rectangles *why* (a + b)** 2 expands to the equation it does but refused to chart out results for various values of a and b?
ii. Would you fail a student who could reliably chart out results for (a + b)** 2 given various values of a and b, but failed to explain that using diagrams of squares and rectangles?
iii. Would you fail a student who could reliably add 1 + 3 or 7 - 5 or 7 * 5 but fail to compute 4592 * 314?
iv. Is a student who could reliably add 1 + 3 or 7 - 5 or 7 * 5 but fail to compute 4592 * 314 at the age of 5 superior or inferior to a student who is capable of doing the same at age 7?
2. If an exam is composed of tables of long division and your score on the test boils down to your ability to how many of those long divisions you can complete in 45 minutes, does the person who scores the highest have a superior understanding of division than the slowest student who suffers from mental fatigue due to an underlying, undiagnoised health condition?
3. When you say "understand how the answer is produced", what level of abstraction is acceptable to be not considered as cheating? Do they need to understand how the calculator physically computes the floating-point math, or is pushing the button enough? If pushing the button is cheating, why isn't using a base-10 shortcut algorithm also cheating? What if the button pusher explained to you precisely how IEEE 754 worked, how a digital calculator works and then refused to do long division citing it was a complete waste of their time and yours? Would you refuse to let them pass the class or fail them unless they yielded to your demands and complied with your specific definition of learning?
Is the math class also doubling as ability to pass compliance and behavioral standards or is it purely a test of mathematical ability?
4. If a student uses an LLM to generate the boilerplate code for a script, but can perfectly explain the architecture, debug the logic, and scale the deployment, have they failed to 'learn how to learn' just because they didn't manually type the syntax?
Educational resources and classroom time is a zero sum game. We cannot afford to educate everyone if they all require 1:1 coaching from a qualified human in a room that has limited space. Time in a day is zero sum as well. Every minute someone spends on doing the 135th long division is a minute they're not spending thinking if solving long division problems is a meaningful differentor in their long term success - whether it's the long division that will make them wealthy, happy and successful or something else entirely?
> i. Would it be sufficient if they proved they understood what addition, subtraction, multiplication and division was? Or do they need to be able to correctly calculate what 4592 * 314 is? What if they could show you using diagrams of squares and rectangles why (a + b)* 2 expands to the equation it does but refused to chart out results for various values of a and b?
I’m not an educator by profession but I’ve done a lot of training and mentoring.
There is no replacement for actually doing the work. People who study something but don’t go through the exercises feel they understand topics better than they do. It’s only when they are put in a position where they have to apply it that the cracks in their understanding are revealed.
This would 100% result in students who thought they could explain multiplication but only had a surface level idea. The knowledge would also be fleeting because actually doing the thing makes the knowledge stick more than just observing and understanding the thing.
So yes, you still need to have the students do the thing.
Replace math with writing and it will be more obvious. It’s easy to look at someone’s writing and critique it. It’s much harder to write well without practicing.
Depends on what skills you want the students to have.
In e.g., philosophy, allowing students to write their assessments with the help of LLMs changes the students’ depth of understanding, as well as their ability to synthesize, formalize, and drum up their own arguments and objections.
These are all useful skills, even in a post-LLM world.
> allowing students to write their assessments with the help of LLMs changes the students’ depth of understanding, as well as their ability to synthesize, formalize, and drum up their own arguments and objections
indeed but LLMs improve these capabilities. Let me explain:
Most educators behave as if using an LLM robs the student of their ability to think. That has not and never been my experience.
You can just tell an LLM to debate, find out inconsistencies and issues in your writing, and challenge you, and it will very well do that!
LLMs for me, cheaply replace teachers who can barely be accessed during office hours because they are so busy doing research or overwhelmed with grading coursework.
In a typical educational setup, I might have half an hour at best with a teacher who looks kindly upon me, and at worst, five minutes with a teacher who doesn't like being around me. An LLM has no emotion, but it has knowledge, it has inference rules, and it has the ability to push back, debate, and find out inconsistencies, bugs and gaps in my thinking.
Behavior that is exactly what I need consistently, in a good teacher.
A frontier LLM is a good teacher who doesn't play games, has no emotion, infinite patience and always available for $20/mo.
There are two kinds of engineers in Finland. Some received a more theoretical education at research universities. Most went to more applied institutions that focus more on practical skills. Regardless of what employers say, they generally prefer the theoretical engineers from research universities. Because the higher status of those universities attracts more talented individuals, because theoretical understanding tends to stay valid longer than practical skills, and because it's easier to learn practical skills at work.
The kids who start their studies today are supposed to graduate in 2030, and they will probably retire around 2080. Focusing too much on the skills their early employers might want in the 2030s would be a huge misallocation of resources.
> because theoretical understanding tends to stay valid longer than practical skills, and because it's easier to learn practical skills at work
depends. If you're building a datacenter and need welders to weld two very specific set of metals together, allocating a PhD in metallurgy to do that welding, I argue would be a huge misallocation of resources.
Now your argument seems to be whether the PhD in metallurgy can pivot to doing something else entirely compared to someone who has been narrowly trained to only weld two very specific set of metals together.
If I interpret your take on the matter, your point seems to be that the former is "intellectually superior and more flexible" than the latter.
I dont take that position - I find most humans to be extremely capable. Just because one human was able to grasp the intricacies of metallurgy doesn't necessarily imply they have more capability than someone who didn't.
The fact that employers in Finland can be so very picky says more about the productivity of Finland than the employee's capabilities. At peak productivity, Finland would just not be hiring anyone who could breathe but import labor because there's not enough people in Finland.
First, employers favor applicants from prestigious schools over those from less prestigious schools, because prestigious schools are more selective. That seems to be at least as true in the US as in Finland.
Second, theoretical education is more useful in the long term, because it's a better foundation for learning new skills. It can be a disadvantage when you are trying to get your first job, as you have fewer practical skills potential employers would need right now. Which is why, from an individual perspective, theoretical education only makes sense in higher-tier institutions, where the prestige can offset the initial disadvantage.
I heard your points - you communicated them very well. I failed to communicate that the first, while true, isn't going to matter in a few years time and the second is not true.
I will repeat what I said - if an employer arbitrarily restricts their talent pool to legacy credentials, they are optimizing for signaling rather than raw productivity.
By 2026, there is overwhelming anecdote and evidence that credentials and where you went to school have very little to do with demonstrated performance. (See citations 1-3)
Hence, it's mostly theatrics.
It seems like you're arguing that it's safer to attend prestigious schools because that increases your chances of being employed. I argue that in the age of LLMs, where the cost of learning is very low and speed of building prototypes (and hence learning and improvement) is rapid, it's easier than ever to start your own business.
> First, employers favor applicants from prestigious schools over those from less prestigious schools, because prestigious schools are more selective. That seems to be at least as true in the US as in Finland
This represents a trailing-indicator mindset typical of "MNC".
Yes, legacy corporations still filter by prestige because they want to signal to others they hire only the best, but that's because in extremely large orgs like theirs, it's impossible for a single person's contributions (outside the C-Suite) to have an outsized effect on that org, so they resort to proxy metrics, like credentails, for signalling.
In real life though, credentials have an extremely weak correlation with impact (see citations).
What this translates for an end user, is that the end user doesn't care about the credentials of the person who built the product. I explain that below.
On the frontiers of excellence - startup founders will tell you that hiring people from prestigious schools is a liability over those who might only have a high school degree because the prestigious school graduate is entitled and is looking to build a resume so they can jump ship in 2 years instead of the barely high school degree student who's hungry to prove they are worth investing in.
The selectivity of prestigious schools is gamed heavily in the U.S where you have literally floors of "prep schools" full of trainers telling you which piano lessons to take, which tennis tournaments to play at, which charities to volunteer for and which SAT question types to learn well. Getting into Harvard is less about ability and more about checking off boxes on a checklist.
Notice that absolutely none of this actually has to do with what the student actually wants to do in life and everything to do with impressing the gatekeepers to admissions.
However, make no mistake, the end customer rarely cares about the credentials.
When you watch Netflix or play the XBox, I bet you don't look up the credentials of the production team or the engineers who built it - you enjoy the content and vote with your dollars - and that's real life.
HN is full of people who don't have anything past a high-school degree but they were obsessed with solving one problem well and now they are multi-millionaires and billionaires because they ended up having an outsized impact. There are far more in that bucket who are comfortable but not multi-millionaires (so you never hear about them).
The money is not the end result but proof of impact and this proves my point.
Let's even discard that, citing survivorship bias. Let's reduce my argument to one fact: engineering/code doesn't care about the author's pedigree - all that matters is whether the system compiles, works correctly and scales under load and a prestigious university certainly doesn't have exclusive rights to those skills.
> theoretical education only makes sense in higher-tier institutions, where the prestige can offset the initial disadvantage
That doesn't make any sense. An institution will hire as per their needs: higher-tier, lower-tier, mid-tier all have their unique needs.
The reason why higher-tier institutions can afford to hire advanced degree graduates is because lower-tier, mid-tier simply cannot afford that baggage: highly credentialed individuals often won't work on tasks they consider "beneath them", the actual "plumbing" of engineering, which makes them useless to lean, high-velocity teams - which is precisely the point the founders at start ups make.
I think you're conflating multiple issues together altough you're not alone in the population that strongly advocates for education as a proxy for ability. They have nothing to do with each other and have a lose correlation.
Here's a thought experiment:
- Kid A (Rural Finland): Has internet access, a Claude Code subscription, and a singular obsession. They spend their time learning how to write a provably correct, memory-safe OS kernel. Through sheer, unhindered iteration and thousands of failures, they master thread management, hardware concurrency, durable scheduling, and modern CPU architecture.
- Kid B (New York City): Attends a prestigious prep school. Their entire mental bandwidth is consumed by a generalized conveyor belt: memorizing 100 years of Eupropean and U.S. history, computing the molality of an acid drop, identifying stereocenters in organic chemistry, and writing essays on what Socrates said to Glaucon. They barely touch operating systems because there is no time left after juggling DBMS trivia, analog design, and quantum physics to satisfy a rubric.
Which person would you listen to or hire when it comes to building an OS?
Now your argument might then pivot to "but what if instead of building an OS, I wanted someone to do algoritmic trading on Wall Street? Ha! I got you there! Kid A knows nothing about that, they only know how to write a kernel!"
Kid A doesn't care. Their singular obsession is OSes and they are world class at it. Their brains aren't clouded by molality and stereocenters - an OS kernel certainly doesn't care about them. If in the future, we had a CPU that worked off organic chemistry, maybe he will learn about it or maybe not and there will be Kid C in the unlikely case Kid A failed to build a trading engine from scratch, fail 1000 times, and learn the math on the fly just like they did for the kernel.
---
[1] Google's Internal Data (Laszlo Bock): In 2013, Google's SVP of People Operations publicly revealed that GPAs, test scores, and university prestige had zero correlation with employee success. They explicitly began hiring more candidates with no college degrees because work-sample tests were far better predictors of impact.
[2] The Dale & Krueger Study (NBER): A famous National Bureau of Economic Research study found that students who were accepted to Ivy League schools but chose to attend less prestigious state schools earned just as much money later in life. It proved that the ambition of the student drives success, not the prestige of the institution.
[3] The Schmidt & Hunter Meta-Analysis: The gold standard in industrial psychology (examining 85 years of hiring data) found that "years of education" has a dismal 0.10 correlation with job performance, whereas "work sample tests" and "cognitive ability" are highly predictive.
The actual Finnish kid who wrote a somewhat successful OS kernel was born in Helsinki, to an upper middle class family with a communist father. He chose to study in the most prestigious university in the country, largely due to his family ties. Because the CS program was theoretical, it didn't take that much of his time. He could enjoy student life and also pursue his other interests. The OS kernel emerged from the latter.
That's what education in a research university is like, when you actually take advantage of it. You don't have to focus on whatever seems useful and practical in the short term, you don't have to compete against your peers, and you don't have to overwork yourself. Instead, you have a socially acceptable opportunity to study whatever interests you, and you hopefully gain the attitude that you can learn whatever you need to learn.
Would it work in Wisconsin, Minnesota, South Carolina, Alabama, Louisiana, Kentucky, Oregon, Oklahoma, Connecticut, Utah, Nevada, Iowa, Puerto Rico, Arkensas, Kansas, Mississippi, New Mexico, Idaho, Nebraska, West Virginia, Hawaii, New Hampshire, Maine, Montana, Rhode Island, Delaware, South Dakota, North Dakota, Alaska, DC, Vermont or Wyoming?
They all have a lower population than Denmark.
All but six states (CA, TX, FL, NY, PA, IL) have less than double the population of Denmark.
Top US universities actually have higher teacher/student ratios than top EU schools, and I wouldn't be surprised if this pattern holds in general, so it would be easier to do in the US than it is here.
The important variable is the staff/student ratio, not the size of the country. With few students per class, or many teachers, this scales fine. I'm running similar exams for my (UK) Masters modules, but colleagues are reluctant to do it for the enormous compulsory survey courses as there are several hundred students in them (vs up to about 50 in the MA ones).
For a large compulsory survey course, a well written multiple choice test should suffice. If the test is well constructed, teaching to the test should cover exactly the required material and be no impediment.
Could you point out a reason why education, which can be standardized at least to a state scale¹, cannot be standardized to the state scale utilizing oral defenses? Also, can be ≠ is. Not everything is a startup lol
You said it can be fundamentally different, and of course it can. But “it doesn’t scale” isn’t an argument by itself. In many areas, scale actually creates efficiencies. You’d need to explain what specifically about the larger scale makes the Danish approach unworkable in the U.S
Can you explain why? As I understand it, you're supposed to be able to talk about the work in which you already invested the mental energy to write. Writing involves much more "brain work" than speaking, so if you've truly written the work in question, you should be capable of speaking about it -- unless you have some sort of disability, in which case, there are probably alternative avenues.
Part of college is growing as a human, and part of that is social interaction.
For the same reason a lot of universities in Europe require you to take philosophy. It's completely unrelated to C++ or data-structures, but it adds to your acumen as a whole.
Not that different from some universities that require you to be able to swim[0].
I only know the Uni system of a few European countries, but generally I would expect Philosophy to be a prerequirement that you take for your entrance examinations, not during studies per se (unless you're studying humanities, of course).
Yeah as much as I like the idea, I know there will be some brilliant and driven students who would have a hard time with this due to temperament. For me, if I’m excited about something, I can talk for hours. Maybe not so for everyone though. Then again, not all brilliant and driven students test well either, so… pick your poison?
Millions of students are now donating to neuralink so they can have stealth LLM answers on the fly. Teachers will have to count the number of load bearing per paragraph.
This is already the case for Master's degrees and above in Denmark. I've been the opponent examining students. We ask Master's students to give US instruction on a topic (drawn randomly on a card in front of them, from a list they know a few days in advance). They do a chalk and talk on a topic for about fifteen minutes with three to five professors being "dumb students". It works fantastically and at the end it's very apparent to both student and teachers how well a topic is understood. Marks are rarely contested and in my experience most students think the process is fair.
Of course, woe betide those with social phobias....
Hi, former student with social phobia and terrified of public speaking.
Today: adult with no problems doing public speaking in front of hundreds.
What fixed me: more public speaking and not having the option of avoiding the task.
I worked in sales for a few years when I was younger before I started my actual career, I went from an introvert afraid of public speaking to completely comfortable breaking the ice with basically anyone in those years (mostly because I liked having four walls and a roof) and that led to me doing technical training/talks to strangers years later.
I still don't really like it but I also realised that dragons don't appear from portals and eat you immediately if you do it.
My missus is much like I was but she never got through it, she finds it fascinating that I can.
The social model of diability is great in some ways (encouraging activism), but the idea that everything needs to be changed to be "designed for" people with disabilities has its downsides. Not all disabilities can be overcome, but to the extent that some can (at least partly), often IMO it's a good thing to have an expectation that most people can deal with the real world.
I also had a sales job at uni, just selling windows. But it was great for making me much more socially competent. Highly recommend.
Tech sales for me :).
I was good at it because I sold the customer what they needed vs what the company wanted me to sell them, it actually led to the same customers coming back and asking for me by name and I got offered a job by one of them - I do wonder what my life would have been like had I taken it but it also led to my entire career as well, my first consulting gig came off the back of a conversation with a customer about a quote they'd had for some development and with the hubris of youth I said I could do that way cheaper (I could and I did, found out later how much cheaper and kicked myself, also a valuable lesson - You don't get paid for your effort, you get paid to save them money, if you are saving them a lot of money you can charge a lot of money).
Was never going to be my career and didn't pay particularly well so I wasn't too concerned about what management thought long term.
I studied my masters in Denmark 5 years ago, before AI. This was absolutely the case - first oral exams in my batchelors many of my fellow students were incredibly nervous. By the end the exam atmosphere was very chill. I definitely improved with public speaking, and so did the other students - including some who had been almost incapacitatingly anxious (throwing up, requiring a lot of convincing from us to even attend exams) and ended roughly the same as the rest of us
Practice helps!
What also helped me is being so knowledgeable about the topic that I didn't need to prepare the presentation much (except for the slides and the general plan). When you know what you have to talk about, it becomes a lot more natural and relaxed for me.
I would be a lot more stressed if I had to learn a presentation by heart and it would probably be a robotic delivery.
So I did quite a few "surprise" presentations, with very light prepared material and a topic I cared about. Got me the practice I needed and I learned not to get "stuck" as if I forgot the next sentence. The presentation always flows!
Now, I have a talk at a major conference, and I'm afraid this won't work there, so I'll prepare my talk better this time, maybe even have speaker notes!
That's part of what fixed it for me. What really fixed it was three things:
1. Actually being very knowledgeable about the subject in question, to the extent that I can offer facts or well-reasoned opinion on the topic on essentially any reasonable question. In other words, not being a junior high school student asked to do a 15 minute speech on a topic drawn from a hat.
2. Not having the idea that the audience expects a correct and immediate answer to every possible question.
3. Not doing every single speech literally in front of 15% of my entire peer group, including all my friends, all the people I admired, and all the people who treated me poorly.
I still hate doing Zoom calls without slides or a screen to share because I can't stand having 20 people staring intently at me. It's just distracting. But otherwise I have no problem at all talking for hours on a topic I know well.
Likewise, I thought I was bad at public speaking and avoided it almost entirely until my last couple years of university. By then I found myself surrounded by friendly, curious peers who were happy for me to nerd about whatever it was I became a semi-expert on. That confidence served me well once I needed to lead/train a team at work.
Doing a presentation on Zoom seems kind of pointless as you lose the connection to the audience. I'd rather just assign a pre-record to watch and do a live Q&A later on. More like a Lecture+Tutorial format.
One thing doctor, and to a lesser extent lawyers, have to do is project confidence when they may in fact not have much. A doctor/lawyer regularly has to quickly become an expert on a subject, or seem like one, often overnight but sometimes much faster than that. Briefing something you are still new to, without getting it horribly wrong, is a skill that must be developed.
I run into this in the military too. "The person to do the ethics/roe/safety/drugs/mental health brief just called in sick. Two hundred soldiers will be here at 0800 tomorrow. Read fast."
> What fixed me: more public speaking
Yup, that's the cure. Worked for me, too.
As someone with lifelong social anxiety. There is the concept of White Knuckling an event. If the anxiety is so far beyond your capability that start to panic, you only reinforce the fear instead of soothe it.
That’s why exposure therapy with an anxiety ladder is important
Right, that’s the problem: for most people this works, but those it doesn’t work for, it can really fuck up. So what this amounts to in expected utility is you need guardrails before forcing things this way.
Thanks for sharing this, a good example that people can grow and adapt. Some need a push and support but it's possible.
The best class I ever took in any school setting was Public Speaking in my first year, first semester in college just to "get it out of the way."
The teacher started off with "you're all going to look like idiots and embarrass yourselves, and you'll have to get over it. Best you do it here now rather than in front of coworkers in a few years. She gave us skills and techniques that I still rely on today, and most importantly, the opportunity to practice.
I was a introverted going in to college, and that class didn't turn my into an extroverted social butterfly in 4 months. However, I have never been paralyzed with fear when required to talk in front of a crowd, and that has been invaluable in my career.
I had an English teacher in high school where everyone had to alternately do a 5-10 minute talk on a topic of their chosing at the beginning of class. They were rarely much good but it was a good exercise.
If kids were more aware how much public speaking is tied into their social and especially romantic life later they’d all be majoring in public speaking :)
If kids were aware of how high they can get in a company with talk vs work, they would think all adults were morons. So your approach has an important double duty.
Yes, yes, yes. Thank you for posting this. It drives me absolutely freaking crazy when people do the "Oh, I'm just not _____," having essentially decided they are not _____, and therefore will never learn or change.
Like every time I hear "Oh I'm just not a math person." It's not that I expect everyone to like math because I do, not at all. But I do expect people to not artificially limit themselves. :/
I second this.
I have always disliked public speaking and oral defense since I was a child, to the point of fairly profound anxiety. But please withhold you woe—these people are, as I am today, adults. Masters students. Coddling them and holding low expectations has proven to be a sort of “killing them with kindness” approach to handling modern youth. People need to be encouraged (and required) to do things they don’t like to become proper functioning adults. I didnt like it, and I still don’t, as it the case with most everyone. But we all grow from it.
> But please withhold you woe
as a side note, "woe betide you" roughly means "you're in for a very bad time", not "I feel sorry for you".
"Woe" is also sometimes used to tell someone to just suck it up and deal with it. In those cases it is a way to make fun of someone for being too sensitive or complaining too much.
sure, but what I was pointing out here is that "woe betide you" is a specific idiomatic phrase. it's usually used as a threat, i.e. "woe betide you if you do that".
I didn't see it as a threat here at all. I think it was just pointing out an underserved part of the population when it comes to this. And I think "withhold your woe" was meant to say that getting out of your comfort zone can be beneficial sometimes (or in other words, that it's not necessarily actually underserving).
I think we're talking past each other here :) I wasn't speaking of whether it was a good or a bad thing, I was just pointing out that "woe" meaning "a misfortune or calamity" is the only meaning of the word that applies to the phrase "woe betide", the more common meaning of woe meaning grief or sorrow doesn't apply here because it's an idiom, not two separate words.
I was pointing it out because the first comment I was replying to was clearly unpacking the word "woe" from the phrase "woe betide" and I figured they would appreciate the linguistic footnote.
Funny, one of you sees it from the perspective of someone who thinks "to those" is other people not themselves and the other does the opposite.
I used to throw up and faint as a youngster, when having to do talks. Was unable to call people on the phone for years. Anxiety is a bitch.
However, the only way out is through. I confidently speak in front of 100’s regularly. I can pick up the damn phone (still a twinge of anxiety, but nothing like it was). The cure that works just seems to be “do it until it’s mundane”.
As an aside, it’s vital from the career perspective. You can’t let fear, no matter how extreme it feels, stop you. It’s a terrible cliche, but still true “no courage without fear”.
If you aren't capable of this, in most careers you won't be able to use your degree anyway. The only other even marginally acceptable compromise is something like the essay portions of what used to be sit down tests with only pen and paper, but longform and supervised closely.
(Note I say this because optimally, this encourages less "supervisory" desires by others later in one's career. If your peers and overseers get out of your way, maybe you will relax more and not feel so anxious all the time. Sorry for the second comment not an edit, testing new HN clients.)
Classic “get over it” condescension. Some people are unable to control their strong physiological reaction to in-person judgement. Go tell professional orchestra players who rely on propranolol to control their stage fright to just grow up and start becoming a “proper functioning adult.”
https://onlinelibrary.wiley.com/doi/10.1002/hast.70031
https://www.thecrimson.com/column/a-night-at-the-symphony/ar...
I have approx 10 meetings a week, usually more, sometimes less. Let's say around 8 hours a week spent in meetings, sometimes with people who hold the power to fire me.
Depending on your role this number can be much higher.
I'd say "get over it", because this is simply a fact of life and you'll have to talk to and present in front of a group of people at some point, perhaps regularly too.
Taking propranolol in this case is, in fact, an example of being a functioning adult, and a solution that also works for oral examinations.
Good luck working in the real world if you can't handle interpersonal interactions or being judged. The coddling has to stop. It's doing no one any good.
Is there coddling. I thought this article was about making students do something their predecessors didn't have to do?
Those are proper functioning adults. Maybe they need meds. Most don’t, but I never said no one does.
Welcome to the adult world. When I get worked up from a interaction, by BP goes into the "holy shit you need to be in the emergency room right now before your heart explodes". And not metaphorically, I have been sent to the emergency room and admitted about a half dozen times for this.
I take my meds so that my heart does not explode from talking to a person. So yeah at some point, deal with it.
Clarification for non-Danes: this post is not entirely accurate. Not all Master's courses in Denmark have oral examinations, and many courses before the Master's level (including elementary school) have oral exams. It is unrelated to the academic level, and instead related to the academic discipline and the size of the course. The latter is in my experience the main constraint, as oral exams are difficult to scale. They usually take 20-30 minutes per student, and if you have a course with 200 students, well, you're in for a long time.
The oral exam format described is also somewhat unusual, in my experience. It is usually the case that there is one internal examiner and one external examiner in the room. More than that would be ruinously expensive. The format varies; usually it's a combination of a short presentation and questions asked by the examiner. Drawing a random topic is very common.
Source: I am a university teacher in Denmark who teaches master-level courses without oral exams, and I have been an examiner for many oral exams at university level.
64 year old nerd with social phobias here. When I was in school, I had to practice oral knowledge tests since grade one. I had my first oral exam in grade six (math) and 2-3 oral exams every year since then. If I hadn't had that experience, I would never have gotten into university. I still don't like verbal communications with people (especially in a tense situations like exam), I'm slow etc, but it's manageable because I had to practice it a lot. I don't think I would have survived in modern "the environment must be made suitable for everyone" school/university.
I’m a generation younger but I concur. I hated presenting and my anxiety was crippling. But the solution for that is to practice more, not to avoid it.
100% and I think this is echoed across the thread. It does apply to many many things that are uncomfortable/hard. You somehow have to power through the part where you suck a lot and don’t worry about it. Mastering that is a huge advantage in life
I have some anxiety about public speaking or presenting to a group. Not debilitating but more than normal, or so I feel. But I find that once I get going it fades. Starting off is the peak of stress. So if the examiners are trained how to encourage a person to start talking, perhaps with some introductory conversation, and when questioning not being confrontational, that might help. The goal is to find out what they know, not how polished their presentation or debating skills are.
Would it help to be interviewed in a room where there is no other person, and you can be seen through a camera and communicate through magnetic speakers?
That probably would be worse, because you aren’t seeing how your examiner reacts and will be imagining the worst.
> Of course, woe betide those with social phobias...
The professors at my grad school had a very amazing tact to get through this as well. They had enough time for specific students who they knew struggled with such issues during the mid semester orals and then had an amazing way of getting answers from them even in the discomfort.
It is amazing how much can be done with soft talking, slow progression and pauses in between from professors who care.
>>> Of course, woe betide those with social phobias....
True but maybe a blessing in disguise. As a shy college kid once (although not with any real disability), being forced to get up in front of people helped me get over it. And I was 10x better after a semester of teaching during a break in my career.
So, I grew up in the former Soviet Union. Starting with ninth grade entrance entrance and exit exams were like that. I remember I had to take university physics exam just to be able to qualify for a consideration. It was fairly rough, but I think in a way, good system because it sifted out the wannabes early on.
Yes, I did Physics, Geometry infront of the committee to explain my answer. Language course was written under a dictation, one mistake would cost you a point out of 5. So you had to be flawless on everything, spelling, punctuation, structure, etc. I aced my physics and language, but got a C (3) in geometry, I fumbled on one question.
Same. The focus on written exams rated blindly and uniformly at the US higher education system always puzzled me. I guess that's the only way to scale things up when you pay professors and teachers living wages?
The blind aspect (I would assume) is meant to cut down on favoritism and discrimination. With the US I understand how that would come about.
Whether it succeeds in that goal is a separate topic.
It does indeed sound great except that everyone was corrupt to the bone
the STEM academic world in the Soviet Union was extremely meritocratic. largely shielded from outside influence and maybe the best talent pipeline anyone has ever invented to produce scientific expertise. Exception being biology as Lysenkoism at some point infected the field.
Corruption affected history or economics because it was ideologically relevant to authorities but math and physics proceeded on its own.
> We ask Master's students to give US instruction on a topic (drawn randomly on a card in front of them, from a list they know a few days in advance)
This is also an absolutely great way to ensure the participants really understand what they’re saying, even aside from ai-related concerns. You could always pull definitions straight from textbooks, and write something which mostly makes sense but that you don’t really understand. But you really can’t teach something unless you do understand it, imo. The best command I ever got on a topic was due to TA’ing
This happened to me back in the 1990s when I was assigned presenting Boltzmann machine to the class. I read up on it naturally, could explain each step in isolation but couldn't still quite internalize how it all came together. But as I was presenting it, all suddenly clicked into place. It was remarkable transition from loosely connected foggy concepts to instant clarity which made me break mid sentence. Once re-composed I couldn't help but notice that the manner of my presentation has changed.
The professor (strict uncompromising man hated by most of the class) then remarked, "now I see you really figured it out".
This is not general to all subjects in Danish universities. I studied physics in Denmark and while the majority of my exams after the second year match this description (though not all), my impression is that this is not at all the norm. Students in the humanities in my dorm, for example, seemed to have an overwhelming majority of several-day take-home written exams (with no associated oral defense).
I expect a lot of the social anxiety can be mitigated by arranging things so that students get a chance to know the examiners and build a rapport with them ahead of time. (E.g., if they’re an instructor then make sure they have office hours and encourage students to go.) I’ve got my mental health challenges but that’s how things worked in my grad school and I got by just fine.
> I expect a lot of the social anxiety can be mitigated by arranging things so that students get a chance to know the examiners and build a rapport with them ahead of time.
That's not my experience. Standing in front of somebody who's been your teacher and you might look up to is way scarier than some rando.
Fairly normal, in my experience, for Masters with a thesis in the US as well. And I have not uncommon group project presentations undergrad as well--in addition to grad school.
If you want to be an adult functioning in society, being able to briefly talk about your ideas with strangers is important.
I doubt the question here is about willingness, but inability due to real medical conditions. Assuming real and sufficiently severe ADHD.
As an ADHDer, I think talks and discussion are way better than written exams. The most common treatments for ADHD are stimulants, but human interaction is a natural stimulant, and the real-time feedback of a human audience is more engaging.
Then maybe you're not suited for the task. Most intellectual professions (keyword being profession, please don't bring up the lone mad scientist fiction thing) require the ability to communicate clearly.
If you're blind, you won't be a racing driver, that's just how it is.
That's how my school did it even for high school courses. End of the year exams were infront of the committee. Same thing, you pull a ticket and have few minutes to prepare the answer. I aced physics course, that was a proud moment but credits didnt transfer when we moved overseas...
This is also the case for many graduate programs in the US as well, but what this article is suggesting is that there are now specific oral exams to defend a student's paper, not just general oral exams.
That’s not entirely true. It varies from master’s programme to master’s programme even inside the same Danish university. Assessment is defined by the corresponding programme board.
Thesis/dissertations do have to be defended, but for course assessment that is not a given.
(I run a course and teach as well on other Master programmes.)
This is truly awesome and fantastic, and I hope it becomes a thing all around the world.
that's interesting, i do something similar when helping my 16 year old with his homework. I ask him to teach me the topic first (it's been a while since i took physics and calc hah). By the time he's able to answer my questions so that i can help him then he's answered his own questions pretty much.
I do this at work too. When doing code review i actually do mostly listening and then some questions. If the developer can teach me the code and i understand what they're doing and why then it's usually pretty good code already.
>Of course, woe betide those with social phobias
I can see this being a problem in the US when it comes to "accommodations" in general. Students with an ADHD diagnosis can claim it causes just about any difficulty and get accommodations for it. The local (top ranked, but everyone's seen all the stories coming out of the Ivies...) college here gives students with ADHD double time on exams and has them take it in a special disabilities proctoring center where it's a struggle to get the staff there to even enforce those time limits. Obviously this also makes it infinitely easier to outright cheat as well.
Not a chance you will get past the "my ADHD makes it hard for me to think about these things verbally" challenge at institutions like this.
Oh, at least in the post Soviet tradition of oral exams, you are given plenty of time to think and prepare your argument before you start an actual conversation with the examining professor. And if you need to think more during the exam to answer a tricky question, the professor just shifts to talk to another student who is ready to present their case for the time being. So it is less of a public speaking and more of a conversation where you can take a pause to think through an argument, or a proof, or a problem.
as someone with ADHD who always finished exams first or was the designated spokesperson for group presentations...
I have no idea why you'd want to be taking a test a second longer than you already have to.
Either you know the material or you don't, anything taking a lot of time is borderline stuff where you're scrapping for points you might not deserve in the first place lol
> I have no idea why you'd want to be taking a test a second longer than you already have to.
The theory was that ADHD students have trouble sustaining concentration, so extra time allows for thought breaks and distractions. The idea was that 2 hours for the ADHD student would be similar to 1 hour of concentration for a non-ADHD student, and therefore the inequity had been partially compensated.
The teachers I know have the same observation that you do: The extra test time isn’t usually used by the ADHD students, but when other students (or maybe more accurately, parents) discover that extra test time was available, most of the high performing students suddenly got diagnosed with ADHD at the same time. They were the ones staying extra long to proofread their work.
For the elementary/junior high teachers I know the schools’ solution was to give everyone the same extended test taking time. Easiest way to remove the incentive to seek out a fake diagnosis without taking anything away from anyone.
Absolutely not saying this is even the most common case but: people “without ADHD” but with a diagnosis can use the extra time effectively.
It is most clear when there is some kind of problem solving involved. An extra 30 min to figure out the trick or plan a detailed approach is a real benefit.
The other case is where there’s some missing knowledge but with a bunch of extra effort you can figure out how to do it. That would be something like re-constructing a self-balancing binary tree from some half-remembered lecture notes.
ADHD is known to be able to affect speed and consistency of output, even of mastered material, and in a variety of ways -- hence "I have no idea..."
Standard diagnostic questionnaires commonly reference this. The last part of your comment reflects a lack of understanding of both human cognition and recall and ADHD in particular.
> I have no idea why you'd want to be taking a test a second longer than you already have to.
After I got treated, I understand why.
Too look for mistakes. Now I don't understand why people don't take the maximum alloted time
Treating many ADHD adult students some did take advantage of accommodations available to them.
Problems with test-taking varied, but common issues were getting distracted by extraneous stimuli in "regular" classrooms. Frequently ADHD students were slow readers and really did need/use extra time. But the biggest problem was test-taking anxiety, often quite prominent, contributing to difficulty with tests.
Interestingly, more than not ADHD students were reluctant to ask for accommodations. Some never availed themselves of this benefit. Others only used part of the options they could use.
In my practice were a number of ADHD adults admitted to nursing school, some having serious academic difficulties and eventually asking for accommodations. The assistance was effective. Notably, none of these student-patients failed or dropped out, all graduated. A pleasant footnote to add is a subset of these nurses developed exceptional skill earning them leadership positions in their institutions.
Reputable clinicians wouldn't give an ADHD diagnosis or support academic accommodations without sufficient clinical evidence.
Part of the problem here is that this college has <5% acceptance rate. Very hard to get into and prestigious. The students who get in have been minmaxing and exploiting every advantage they can for their entire life.
> The students who get in have been minmaxing and exploiting every advantage they can for their entire life.
Not a school I'd be tempted to apply to. Scholastic competition is one thing, but scrapping for every conceivable speck of "I'm better than the next guy" is not a bit attractive. I wonder about the campus atmosphere. Seems likely contesting would be a hard to break habit. I feel tense just thinking about it :/
Of course, ADHD students are less likely to achieve near-perfect grades than non-ADHD peers. Though it's not impossible for an ADHD student to succeed. Anyway, not at all uncommon for a student to do very well until subjects become more complex and challenging. Conceivably a student could look promising, but then suddenly have great difficulty keeping up.
So a safe bet that at least a few students at the top-level school have ADHD. If the school offers little or no accommodation, affected students probably have to accept getting educated elsewhere. Fortunately, more than not ADHD treatment is effective, a favorable response can make a pivotal difference in the student's outcome.
> college here gives students with ADHD double time on exams
No offence, but this is a ridiculous thing to get worked up about. Various neurological conditions exhibit in people just being “slow” in exams. No one ever diagnosed me with anything (probably too long ago), but teachers somehow understood that I was one of these and gave me extra time on exams.
I didn’t want it (who wants to be in an exam a minute longer than required!). But I did need it through high school. In university I was able to complete in time (mostly); I suspect because I had practiced doing exams so much by then that I’d figured out how to optimise for time (maybe everyone knows this and I was just very slow to learn it?).
I’m someone who excelled academically, and it was an important part of my identity at that time. And I do remember exams (before teachers gave me more time) where I’d get a lower grade as I’d only completed 70%. It was incredibly painful; I knew the material, if only I could get it down on paper in time! (I’ve also been an incredibly slow writer). Extra time for someone who struggles to complete on time isn’t cheating - it’s allowing people to demonstrate what they can do.
> teachers somehow understood that I was one of these
Are you sure your parents weren't involved in the background?
> Extra time for someone who struggles to complete on time isn’t cheating - it’s allowing people to demonstrate what they can do.
For me it becomes cheating when its no longer allowed to be mentioned on the certificate that extra time was given.
European universities also have accommodations and extra time etc. And yes it's an arms race. Though at some point it will simply inflate away the value of the degree, but educators will stand there baffled, not understanding how it could happen.
Welcome to the real world. One cannot be coddled forever.
Best part you can trivially automate this using AI
This is how higher education was done for literal centuries before moving to the written word. So it’s not exactly innovative to return to oral arguments.
But what it does threaten to do is effectively abandon all the efficiencies of the written word.
When higher education evolved into a mass system in the 1800s and into the 1900s, pure oral examination was looked at as super inefficient. A written paper could be handed in and graded without lining everyone up in front of a panel.
The oral system scales poorly. One examiner can only hear so many students in a day. There are damn good reasons Medieval universities were small and modern ones are not.
Requiring every major paper to be defended live reintroduces the bottleneck that written exams were designed to escape.
Accessibility takes a hit too. Students with speech differences, anxiety disorders, hearing impairments, or simply less fluency in the language of instruction face an extra barrier that a written submission does not impose. The written form, for all its flaws, leveled some of that ground. Shall we abandon the differently abled because machines disrupt the academic system and essentially make cheating effective? This does not seem very logical, or sensitive.
So while it’s nice that academics are rediscovering the old ways, I don’t really see this as a scalable answer to raising education levels for the masses, but more of a knee jerk way to try to bypass the obvious technological disruption by returning to non-scalable, inaccessible ways of doing things that our ancestors abandoned for a reason.
It has its place for graduate students and smaller volume education systems. It’s fine for keeping the ivory tower clean. But it’s not the answer to expanding education to everyone and taking advantage of the technological disruption to advance education and learning throughout greater humanity, which is what AI definitely possesses as a possibility.
Oral exams are still common in Germany and many European countries in case of smaller courses taken by 20-30 students in a semester, and sometimes even in larger courses, and the task of examining is then split among multiple professors and lecturers.
In Hungary it's a pipeline. You go in a room, you pick a piece of paper out of a hat, which gives you some task or topic to prepare for. Then you sit down at a desk (the room is filled with about 15 students working already), then you solve the problem or sketch out your notes to present the topic you picked for like 20 minutes, then you go to the professor and you have to, in low voice, explain your solution, the professor asks back and pokes at it to make sure you understand. The professor then gives a grade immediately and you go home.
In Germany, it's usually no random pick of a topic, you just go through the entire semester's material over 30-45 minutes and the professor stops to spend more or less time on each topic to explore your knowledge. You can explain things by making sketches on paper. A PhD student sits there quietly taking notes of what was asked and what the student answers approximately. You leave the room, the prof and the PhD student discuss the performance, decide the grade and you come back, and the professor explains the grade and what you did well and not well.
I'm not familiar with the German universiry system, but I think a 1 hour, 2 professors (or one professor and one PhD student) would be impossible to scale. A class of about 100 students could take an entire month! (or better, 2-months of professor-hours!)
One big difference I observed between the US and $HOMECOUNTRY is that US colleges require a lot of courses with little credit, so a normal college year is N courses x K credits per courses, N large, K small. In $HOMECOUNTRY it is typically N small, K large - think instead of having Calc IV, Trig whatever, Algebra 2 and Introduction to Proofs, you have a single "Mathematics" course that last a whole year, not just a semester.
Since each course needs its eval, assessment is a function of N, not N*K or K. Much easier to scale if there is ~5 evals per student per year than ~20.
But that is how certain exams are done right now. Though it's usually 30 minutes including overhead. It's not something you can do in classes with multiple hundred students, but it's certainly done at scale.
I was the PhD student in the room in dozens of these exams, and I toke some of them myself. But that was for classes with a bit less than 40 to maybe 60-80 students. The load for the larger ones was shared among professors, if I remember correctly. But it was certainly the case that a single professor spent 1-2 days doing nothing else but these exams.
And it was 1 professor and 1 PhD students, and those don't count. They essentially grow on trees and are always available, if you need some.
> would be impossible to scale
I think education is something better scaled horizontally than vertically. More schools, with more professors instead increasing the student per professor throughput. The classes where I learned the absolute most, and had the best experience, were closer in size to 25 than 100.
The US has proved it can do horizontal scaling, as the majority of adults gets a college degree.
The horizontal capacity is ineffectively used by having many degree mills and low-value degrees, but if we re-tooled the knowledge sets we could raise the bar.
I actually think of degree mills as being vertically scaled. Online courses, a lot of course work graded by computers, large classrooms, etc.
As for the quality, I think small classrooms with good teachers probably produces a better outcome than large classrooms with great teachers.
At low student counts, like 20, (think specialist niche elective master's courses, not the huge bachelor's courses) it's perceived by profs as more efficient in fact than the ordeal of organizing a written exam.
With an oral exam, the prof blocks off 3 days or so, with ~8 students per day and it's done. It's not a huge additional effort compared to all the effort of giving the course over the full semester anyway.
An added, though minor benefit is that the professor might decide to offer a student assistant job to someone who performs well, because it's much easier to judge who is actually good in an oral exam vs in written. There's a reason that job interviews aren't just tests filled out on paper but involve personal interactions.
What is the point of this reply? GP already specified oral exams are mostly reserved to courses with smaller numbers
Oral exams can also compensate a lot for different types of exam anxiety. Blanking on individual points doesn't matter as much, the examiner will help you along or ask along a different path. Doesn't help for someone that completely shuts down, but it's much easier to determine if a student was just stuck on some minor point and actually knows the topic in an oral exam.
They are likely prone to more biases than written exams, and the variability among examiners is likely high. My experience is both in taking these exams myself, and also sitting in on them as the note taker many times. Though that was only with one specific professor, and almost always on the same course. But that was in my experience a very fair exam, even for those students that were very noticeably nervous.
I think you have the right observation but the wrong conclusion.
The appeal of digital systems, at a social level, is that they can transmit information accurately for almost no marginal cost. They come at the risk of fraud.
With AI the levels of risk have increased. Anyone can now be a fraudster, and it gets harder and harder to prove anything online. Either of us two, in this conversation, could be a well tuned LLM.
So society, collectively and at an individual level, is forced to reassess reliance on purely digital information. Which means that the advances in the technology, actually end up reducing the utility of digital systems more generally.
You can argue that this makes education costly and less accessible. It does. It absolutely does. There is also no alternative except to see higher education get wiped out, which would have devastating economic and social effects.
So we are going back to blue books and vivas.
"So we are going back to blue books..." exactly. And, it is also efficient. You just need a proctor to ID the test taker and make sure they aren't using AI to cheat in order to verify that the student learned something which seems like a good idea.
Even in fairly recent times viva voce was common. I had to defend my final year report on a long piece of practical work as part of the final for my BSc in Applied Physics in 1975.
You don't need to do everything orally. A return to handwritten final exams done without electronic aids would help too. They can be open note as mine were in the mid 70s or even open book.
As for scalability, I really don't see the problem; if we are producing so many qualified people then we surely can use some of them to examine the next generation. It sounds like the argument that often appears on the web where someone, usually an American, claims that a solution that works in, say the UK, can't work in their country simply because their population is larger.
Also, we really should consider if shoe horning every young person into an academic institution is useful either to society or to the person concerned. It certainly has not been an unmixed success in the UK.
I'm quite skeptical that industrial-age measures of efficiency are the right way to think about education. Feinman's story about teaching in Brazil comes to mind: https://enlightenedidiot.net/random/feynman-on-brazilian-edu...
And that's before we get to technological changes making that sort of knowledge and those skills much less valuable.
As an aside, accessibility cuts both ways. As a person with ADHD I'd much rather do oral and practical exams.
I graduated in 2000 and remember doing the oral defence. It wasn't a huge deal. You already have a personal relationship with your tutors and they are familiar with you and your communication style. They are not the Gestapo.
This is basically my thought- my school is considering all sorts of things to combat AI cheating, and most of them scale poorly. You can implement them and increase your reputation, but your tuition will have to increase to pay for a lower student/teacher ratio.
So we’re probably going to see a split- mass production schools that still act as job training for Industrial Revolution era jobs will keep doing what they are doing. Schools that want to keep their academic prestige will implement these inefficient safeguards and raise prices accordingly. Employers will distinguish between these two types of diplomas when there are enough options out there that are combatting cheating- much like in-person schools are generally valued higher than online educations, or how certain schools (MIT, Stanford) are valued higher then others. This might lead us to the transformation of higher education many have been anticipating, because the current system cannot remain as-is. We all thought technology was the thing causing the upheaval but AI has really accelerated things.
And what a better time to set optimization aside when schools and universities are getting less and less students each semester due to population decline.
Interestingly, you dont consider your 'inverse' point: that now the manual part of teaching the stuff can (and definitely will) be automated with LLMs.
It will become similar to coding: humans evaluate, LLMs just provide what has been mechanical for a couple hundred years
> A written paper could be handed in and graded without lining everyone up in front of a panel
It can also be graded blindly. That allows for removing–or at least mitigating–biases in evaluation.
It’s a good answer for keeping online education honest. It doesn’t need to be applied to all in-person schools, where teachers distribute spot checks across the course relatively evenly, but it makes perfect sense in conjunction with online testing. You only have to challenge one essay per student per term to catch cheaters with a near-perfect success rate. Assuming a 12 week course with 36 students, that’s qty.3-4 of teacher:student one on one meetings per week for the term; I think that’s doable. If you’ve oversold your teachers and have more than 36 students per, guess you’ll have to increase your labor expenses :D
It shouldn't take long for educators suss out whether students know the subject they are talking about. You don't have to question them exhaustively to determine that they likely did or didn't do the work they are claiming. TFA is about high school projects, not PhD defenses.
Complete bullshit.
Universities in the US charge fortunes and pay instructors a pittance while plowing ever more money into “administrators”. Fire half the administrators and hire more instructors to both proctor exams and give more personalized instruction.
> Students with speech differences, anxiety disorders, hearing impairments, or simply less fluency in the language of instruction face an extra barrier that a written submission does not impose. The written form, for all its flaws, leveled some of that ground. Shall we abandon the differently abled because machines disrupt the academic system and essentially make cheating effective?
Universities are entirely elective, why would they need to cater specifically to disabilities? Driving schools don't make exceptions for the blind. There are skills that people expect you to have with a certain kind of diploma, and it reflects poorly on the school if they don't have a standard to hold people to. Not being able to present your work in a basic way is one of those mandatory skills that imo makes you unqualified to graduate if you don't grasp it.
I'm not saying you can't make an exception for a person or two who is clearly as good or better than the rest otherwise, but people will always try to misuse as many exceptions as they can, en masse, as an excuse to just avoid doing required work.
I feel this was already a severe problem pre-AI. From what I hear from my partner (in university administration), nearly half the students demand accomodations for disabilities. Many of those conditions are very difficult to falsify
It doesn't feel good, but at some point we have to ask, can we support every single case, at the expense of the whole? I'd like to be convinced that's not a necessary tradeoff
What I don't get is why we disadvantage everyone who's honest. Just give everyone extra time to level the playing field.
I don't think it's that bad yet, but it's certainly on its way there. We're like at this intersection of lots of conditions that previously weren't documented becoming actual diagnoses, that pulling along a lot of peer-pressure kind of trendiness to have some kind of fault that makes you more unique, and the practicality of a system that was designed to give massive benefits to rare exceptions, for which now so many easily qualify on paper.
The list of conditions that qualify for what will most likely have to be redone completely from scratch.
I don't think the world really works like that. Hearing an elevator pitch has a very different impact from reading a short-form business proposal. And of course the efficiency of the written word is not absolute; in lively discussions and debates, there is no time to put thoughts down into the written word; they had to be uttered. I am personally of the opinion that a successful university graduate should not only be able to write down their ideas in words but also be able to speak to defend their ideas.
"The oral system scales poorly. One examiner can only hear so many students in a day."
I'm not joking: you could have an AI quiz the students and test their understanding. That way the tests go deeper into testing the understanding by following through on the answers, and are standardized and fair for everyone. I don't think we are there yet at all but I actually think its not insane for the future. I expect downvotes though/I don't think our generation would agree with this.
What's old is new :)
But even with the written exam, wouldn't the bottleneck still be there where the professors still need to read through all the papers? I guess its definitely more efficient and asynchronous, but I can't imagine by much assuming the same and equal rigor is applied.
Grading written exams is delegated to teaching assistants / PhD students.
Not everywhere.
You can be certain many instructors are having AI do a lot of their grading.
As an educator myself, this quarter for the first time for my course's final project, I've switched to actually asking my students to provide an "AI Authenticity Audit" of their (project-relevant) chats in the construction of said projects. All my students' output over the last two years went from pre-AI level quality to boardroom-ready polish, and I'm actually more interested to see how they arrive at an outcome rather than the final output itself - focusing on their questioning, curating, editing, and iterating process. Didn't think we'd get to somewhere like this so soon, but here we are. Strange, but I'm trying it out.
IMO this kind of thing adds work and stress for the best students. If I'd been faced with something like this while I was in school I would have felt the need to basically curate a parallel construction AI discussion, knowing it would be judged. I understand the theory that, in the age of AI, the prompts are more interesting than the outputs, but I'm not convinced this is the way.
Things like the Denmark approach (and/or in-person written responses) seem like a better direction. Treat take-home writing the same as we've treated take-home math questions - as useful practice (for those who choose to actually engage with it), but not a significant input to grading.
Yes, I thought about that, and which is why I'm just running this as a pilot for this quarter to see how it goes and I'm not particularly attached to it if feedback/results are mixed. But that said, there are real constraints on my ability to supervise students with in-person examination, but that is more a problem with how my society funds and cares about education. We are forced to take approaches that 'work at scale', hence experiments like this. But truly, reception was positive when this was introduced, because students too want to be held to a good standard and don't want others to get credit for not doing the work toward achieving the learning outcomes like the most honest ones.
Oral Defences have a long established tradition in Denmark, but recently there’s been real cut back on this as a money saving exercise. So both teachers and Students are familiar with this form of examination.
So for a dane this reads like “back to the old way”, rather than anything new or novel.
I'm quite impressed with the Hungarian system, for 18-19 year old school leavers, where every subject is split 50/50 between:
The oral (szóbeli) has:In my high school English class (like 10 years ago) my teacher assigned readings and called on a random person each day to give an overview of a section. He basically quizzed you in front of the class on themes, plots, etc. It was very effective at getting me to read and care for that class, because if you just sat up there and didn't know anything you'd lose marks on participation (and look kind of dumb). Not sure if/when/why oral presentations and public speaking are no longer got prioritized.
I had to do an oral defense of my thesis in graduate school. When you've spent months and months writing something extremely obscure... well, I'd hope that it's not that difficult.
You suddenly realize the foremost expert on the research in your dissertation…is you.
Yes!
A fun anecdote from my defense was that this element of logic "did not exist before some time" or something.
One of my citations was a book from like, the 1700's or something, and it absolutely blew away one of the committee members that I'd found it. He asked me "how on earth did you find this reference?" and I just told him that I look at all the math/logic libraries in Boston I had access to, I looked up all their logic books that predated the cut off I was looking for, and I skimmed them.
I didn't even have to leave my university's library, because one of the first five books had the counter-factual I was looking for. Sure, it was a rare book and I had read it in a special room, but it wasn't particularly hard to find.
Before that defense, I was 100% the only person alive with that information.
The problem with oral exam is that it’s done serially student by student, which can be infeasible with large classes. The grading is also highly subjective.
Written exams are also not best suited to project-based classes.
What’s the best way to grade students in classes with coding projects?
Its serial nature also means that an evaluator's standards can drift from students with early exams to those with later exams. Oral exams can introduce issues of sex and ethnic biasing.
Evaluation drift happens with written exams too. They are graded serially.
Or just regular personal biasing, which is just as unfair but doesn’t receive government protection even in theory.
Arguably, the problem with higher education is one of volume.
Why is everyone in this thread comparing this to a University
But yeah, this is a good step.
Homework is also becoming obsolete at this point. If they can't do it with pen and paper in class, there's no reliable way to evaluate work anymore.
I'm teaching a university math course next semester and I've been messing around with a solution to this that my old analysis professor used to employ. Every week, they take home a handful of theory questions. These aren't turned in for credit. But, every week, there'll be a short quiz where one of those questions are asked verbatim, along with a small follow up.
That felt like a very natural solution to me. Of course, worst case scenario, they just blindly generate and memorize the proofs without thinking. But I think the follow ups will catch them in that case. Moreover, I choose to believe that >90% of my students are willing to at least put in the work of scrutinizing those proofs which is honestly not bad exercise in its own right.
I would love to do oral exams directly, the logistics with the number of students I have just make that hard. I know we're trialing it with the less populated special topics courses.
This follow up idea is a fun one! Not likely I'll have to be teaching a course like this again, but I will suggest it to some people I know.
Only as an honor system for students who want to learn the material.
But worthless for assessments and assigning grades.
The Conversation has an article that supports raising expectations for students, given that they have access to AI
https://theconversation.com/mit-researchers-say-using-chatgp...
I wish we were doing this in the US even before AI was an issue. I suspect it would reorient students to their work in some pretty cool ways. I mean, it might feel like torture for some and maybe that’s not so much a bad thing. I’m sure I’m not thinking of some downsides to this too, but it seems like such a great counter against “factory” schooling. Of course, this could be a lot more work for faculty and could be hard to keep up with a huge student population. Again, maybe a feature more than a bug.
It was already long-standing practice, when I was teaching many many years ago, to interview suspected cheaters (it's usually quite obvious) to see if they actually knew what they claimed they knew.
Only the dumb ones are obvious. In my school (SFU/BC) most of the students in almost all of my classes cheated. VERY FEW got caught.
I heard instructors say all too often that it was obvious when somebody was cheating, but somehow they missed the guy paying his cousin to write the final.
Sometimes it's just an empty threat, and instructors don't want to deal with the rest of the fallout.
Even more so these days with the infusion of identity politics in education.
I should add, this was for CS. I don't know about the other programs.
My entire university education was evaluated mostly by proctored closed book exams and oral defenses. AI proof 10 years before AI.
This just feels like good educational practice too? I remember really enjoying TA'ing as an undergrad for courses I took the previous year, because THAT'S when I felt I _really_ learned the material.
Likewise, when I was a lab consultant, I felt like advising others was when the concepts of Linux/systems administration _really_ sunk in.
I always crammed the bare minimum for any kind of assessment (by the end of college I was actively avoiding any classes with final exams), but when I need to be able to teach someone else or explain my understanding in a live environment? That's when I _actually_ learn.
It feels like systems like the Maturita' final multi-day exam in Italy do a pretty good job at providing students with a standardized, state-supervised process for demonstrating one's competence by the end of high school. No computers, you're under the watchful eye of your teachers the whole time, nobody knows what's going to be on the exam until the day of.
https://en.wikipedia.org/wiki/Matura#Italy
It's time consuming and nerve-wracking, but it feels fair.
Side note/ anecdote.
When I was in school there was a maths aural test. The teacher read out questions and you wrote the answers, with no calculator. Basically a stupid name for a mental arithmetic test. The funny thing was the teachers were so at pains to differentiate between two words that sound basically the same in our local accent, aural and oral, that they insisted on pronouncing aural as "ow-eral' the ow being pronounced like a pain reaction sound. The smarter kids took to teasing the teacher with words like aura and aurora, and asking how to pronounce them...
Fun story: Final oral exam (in history, I believe -- a thing back then), I took a whole A4 paper and turned it into hundreds of little balls in front of 3 teachers over the duration of the oral exam.
I had no idea I was doing this until one of the teacher pointed to the heap in front of me, but it saved my bacon as I did otherwise well.
in the french CPGEs they have you make a mini research project to be presented in front of 2/3 judges at the end of the 2 years, each year has it's own theme (this year it's efficiency/optimisation) thankfully it doesn't need to be very deep, but it has to tie somewhat into the curriculum which can be annoying
I wrote about assessing students in the era of AI from when I was teaching at Carnegie Mellon: https://austinhenley.com/blog/aihomework.html
Good to see.
Related, I've moved to requiring regular in-class handwritten quizzes. Just a blank piece of paper to start, with the questions on the board. The practice forces students to recognize whether they are learning specific topics.
This is how we used to grade labs when I was a TA in college. Students worked in groups of two and showed up with a working project to get graded. Afterwards, we'd ask the two students questions and it was extremely obvious if one or both of them didn't work on it.
Oral exams were the norm when I studied. Even for technical topics.
As a thought experiment: In-person testing or oral defense approaches work until we get implants. If people have implants, what then? Demand that the implant be turned off? Demand that the implant's data connection be turned off? In the latter case, the person with more storage or a better local model wins.
In the long run, in the AGI world, if everyone has implants, then everyone is roughly equal because computer-aided computation will be better than what any individual can do. Those without implants are screwed.
(I don't want an implant; nor do I want others to have implants.)
Thos whole debate is happening because people think a degree matters. Because getting a piece of paper is more important than knowledge. Students will continue to cheat (AI or not) as long as the goal is to get a stamp.
I’ve studied CS in Italy and most if not all exams were both written and oral some also had labs. I thought it was the same for other degrees too.
In my uni years bachelors, masters and PhDs required thorough research resulting in a huge opus printed and stitched together but the only thing that really mattered was the presentation of the work in front of the committee and oral defenses. To this day I have been living in full confidence that this is the academic standard that has been in place and effective (with variations) over centuries, if not millenias…
Even intermediate exams were a combinations of practical work in some written form and the theoretical grill sessions in the end of every semester.
All in all, if I had to put a professor’s hat today I would not give a sh.. if my students would use LLMs to write text. What would matter (and what does for me as a hiring manager at $WORK) and a human being with decency is the results of the _natural neural network_ training so to speak. IF whatever prompts and interactions with LLMs result in sharp knowledge of the subject and ability to apply the knowledge to solving real problems, well done and congrats!
This is a great strategy. It is something I do in code reviews as a senior developer. There has been observed longer code reviews for seniors and this kind of approach keeps it manageable.
This is great. Sorry, but the Western World has to stop normalizing extreme introvertism. Humans are social. We rely on communication and cooperation to survive. Allowing kids to shelter themselves from interaction and communication during crucial development stages will only hamper them in life.
Couldn't students ask AI to make a oral defense plan?
Did you know that there is a foolproof way to cheat on any exam? You can just store the answers in your brain. They won't even notice you cheated.
maybe the policy will succeed in encouraging learning for most students (at least I hope so!!); otherwise some might still rely on ai because of the trust they've built up with it, a lack of connection to and therefore faith in the info they're expected to just memorize, low confidence in their abilities, etc
Woah! Really! I never knew that!
Of course they could, in fact it'll probably be great prep if they read it and understand it instead of just memorising the answer.
If they just memorise the answer any smart teacher can trip them up by just rephrasing things or developing tangents to see understanding.
In school before exams, I'd often make little cheat sheets about things that you'd need to memorise like formulas for areas and volumes or certain physics equations (we weren't allowed any reference material for that on exams). The very act of writing them all down before the exam would often boost my memory enough to the point I'd just recall them.
At this point you're just describing learning using buzz words.
I find it quite surprising that Danish high school students (the article isn't about university) apparently did write digital exams with internet access until now, so basically open book.
I'm also surprised. In Portugal primary, middle and high school tests and exams are all proctored and the vast majority of them written by hand.
Brillant! <- I had AI write that.
You've touched on something real here
I mean, this is pretty much the easiest way of handling the problem.
When I was teaching in a university the easiest thing to test someone's understanding was a simple question: "Explain me how would you use this in a day-to-day situation."
If the guy knows his business, he'll go on a long lecture obsessing over every new feature of C# 2.0 or PHP 3. He would give you all the differences between Firefox 3.5.1 and IE 6 with Maxthon. (Goddamn I'm freaking old)
Should you not find a spark of interest but just random mumbo-jumbo of words that don't constitute sentences, you can clearly see that there is no understanding present in the subject.
In this case you work as a teacher, helping the guy to figure out what he does not understand in a subject and simplifying things for him till this clicks. After that he will be able to talk about said subject for hours with gusto.
Somehow Russian technical universities were never relying on written work, there is some, but even that is is delivered in person with the tute teacher asking this and that.
Tangentially, the market for uni assignments (mostly humanities) for pay was huge before AI.
A nice plot twist would be if those tests were administered by an AI.
More like a tired trope.
At some point they will have to be won't they? Too many students.
At scale, I imagine something similar to the Baseline Test in Bladerunner 2049.
One step closer to mentats.
Watching tech cannibalize itself is about as entertaining as watching Hollywood do it in the mid-2000s.
From my conversation with some undergraduates, 99% of people are using LLMs for any applicable assignment.
That's going to be a hard test. Have you ever heard Danish?
In slovenia, at least in most engineering colleges, every exam was a combination of written + oral, way before AI even existed.
This is and have always been the case for Masters and above across Europe. It varies for Licentiates.
It is also often the case for exams in majors (they are often oral).
So nothing innovative there
In my part of Europe oral exams have been a staple even in highschool and across unis. I was pretty surprised when I've seen that other parts of the world don't complement written testing with oral understanding checks.
This is a good move. I teach a summer async remote course that's ending next week, but I'm giving all the students incompletes until they come back to campus and take an in-person exam the first week of classes. All homework is worth a marginal amount, and if they fail the final they fail the class.
I'm not going to spend my effort policing whether they're using AI or not in their class work because it's a losing battle. At the end of the day, they can either demonstrate mastery of the material at an exam or not. It makes the exams higher stakes, but that's where we are.
I hope at least the homework is good prep for the exams, otherwise if its grade value is marginal there's little point from the PoV of a student that just needs to pass the class to take it seriously
I'd never pay to have you as a teacher, I'm sorry. Maybe your move makes a sense for the reputation of your university's degree, but as a student willing to pay to learn remotely, I'd not accept to be treated like children.
Regular graded assignments is the "treated like children" version. The adult treatment is that you can come to class if you want, you can do the assignments if you want, but if not, that's fine too. There is one test at the end, and it's your responsibility to prepare for it as you want. You manage your own time, we offer consultation, lectures, assignments, tutorials etc. but you can reject all of this and learn the material from some other textbook, and as long as you've mastered the material, you'll get a good grade at the exam. This is how German universities work, for the most part.
The problem is the vicious feedback nowadays of online student evaluations of professors. Professors that are rigorous get complaints and low rankings, kids don't sign up for their sections etc. Kids (parents) pay the administration and the administration wants to keep the customer happy. The only real answer are standardized exams (such as licensure exams) which are beyond parent pocketbook capture.
Or like Europe. You don't pay tuition, but if you slack off, and don't progress properly, you only have a limited number of exam retries and limited number of funded semesters. If you fail, you're out, choose another profession. You're not a customer, you're someone who is trying to prove themselves and get certified by experts on their knowledge. The "customer" of the university is whoever will glance at the degree paper and tries to ascertain the skill/knowledge level of the outgoing student. They have to give a good signal there. The reason this doesn't work in the US, is because they want mediocre children of rich parents to be pushed through, because college is a networking and social class-based finishing school, not a place for acquiring knowledge.
(Also, there should be electives, but the rigorous backbone of a degree should be mandatory, so "students won't choose this course" doesn't come up.)
> The reason this doesn't work in the US, is because they want mediocre children of rich parents to be pushed through, because college is a networking and social class-based finishing school, not a place for acquiring knowledge.
Out of curiosity, which US university did you attend? I went to a well-known and highly-regarded public university in California, and they absolutely did not coddle their students. Poor-performing students regularly washed out.
None, but have interacted with American students at European universities and they were very lost without all the coddling and all kinds of support infrastructure and basically getting their life organized by administrators who personally attended to their matters and reminded them a million times about deadlines and so on. They were unprepared for actually having to manage their own studies, read up on rules, find out where to register for something, until when, etc. etc.
That happens when American students first join US universities too. It takes a few weeks for freshmen to get acquainted with the new reality of being completely responsible for handling their affairs and ensuring their success. It’s nothing to slam them about; that’s an artifact of coming from a system that provides more guide rails during primary education and childhood than European children might enjoy.
American colleges give way more support throughout the studies though. In Europe you're literally alone and nobody cares if you miss deadlines or you're failing out. No general consultations, no extra departments and admin bloat. You have to have your shit together.
It really depends on the university. Mine wasn't particularly helpful in this regard. They sent me a form-letter warning by mail once, when my GPA sank too low one quarter (this was before email was ubiquitous). And they did have some counseling resources available, but the resource-to-student ratio was very, very low. I would not call it white-glove service by any means.
Perhaps you also don't have a comprehensive picture as to what is available to students across every university in Europe. Europe, like America, is a big place and, like America, the situation probably varies by institution. As with most things, the answer is probably more nuanced than general observations would make it seem.
Tenure protects professors against students' complaints (often for the better, sometimes for the worse). Also, the administration already knows that students complain about professors who are tough, and don't typically pay heed to those complaints. Parents don't really have recourse either: what are they going to do, send their kid to Podunk State instead?
Yes indeed. College departments are well-known for cutting the ranks of the professors who have proven track records of boosting enrollment in favor of awarding tenure to the professors who can't get butts in seats.
I don’t think you understand how universities work. Which US university did you attend?
I don't think you understand how hiring, promotion and departmental finances works at US universities for roughly the last two decades.
I am very well aware, having close relationships with multiple faculty professors in my family to this day. Practically none of it has anything to do with student ratings.
Which university did you attend? I asked you once already; put up or shut up.
Why does which college I attended have more bearing than the ones I have and do work at? I don't particularly care for your stupid irrelevant demand and I am refusing to engage it.
I, my parents, my in-laws and my siblings are all academics. I don't need to prove anything to you. My brother is faculty at an ivy and literally what I am describing is what is currently going on in his department because the administration has foisted MBA brainrot "performance metrics" that are easily gamed by students and faculty who don't particularly care to teach but need points in their portfolios.
Students are entitled "I'm the customer, and I'm always right and I'm paying you a lot" twats who get the administration to bail them out if they don't get As for half-assed work. It's similar but not as bad at R1 state universities where I work and have worked.
You are completely out of touch and which university you or I attended in the 90s (two generations ago) is completely irrelevant.
Is your brother actually a full tenured professor? Does he bring in grants and endowments and is well known in the research community? Is he being threatened with serious punitive action for not giving into individual student demands? There is a tremendous difference in treatment between those who are fully tenured and those who aren’t. Lecturers and adjunct professors are frequently treated differently, and I accounted for that in my original reply by referencing the tenure track.
Students can ask for whatever they want. It doesn’t mean they will get it, unless maybe they are VIPs at Ivy League schools, and those are a sliver of the total population of university students. Don’t mistake the minuscule minority for the majority.
You’re letting your personal experience cloud the full and diverse picture of reality. Consider that we may both be right depending on the circumstances.
I don't give a shit what you "accounted for", you're hot air with no teeth in the game passing yourself off as some sort of "expert" or knowledgeable about anything. I might as well talk to a LLM.
You’ve gone fully off the rails and are making things personal. I’m done with this thread.
I appreciate you and that you still believe the lies of how academics works.
If you only care about the learning, then you could just take the incomplete. You don’t somehow lose the knowledge you gained.
The only cost of an incomplete is it doesn’t help towards a degree, but a degree is just a signaling mechanism. If you want to be able to use their degree to signal something to someone, you will have to jump through their hoops.
Adults don't have to prove things?
As AI eats up the need for half-assed "I googled a few things and wrote something up and then slacked off for five hours" white-collar-paperwork jobs, it's gonna be pretty hard for people not willing or able to demonstrate that they can do anything that couldn't just be fully done with an "agent" instead.
> I'd not accept to be treated like children.
What is your expectation from such a class? What are you hoping to get out of it?
If you merely want a tutor that will help you out navigate a topic you are interested in, with no strings attached, then sure, yes. No need to take exams.
But if the goal is to get a formally recognized degree in some topic, then you will need to accept that there will be oversight over how you understand that topic. Because the institutions granting those degrees need to sign them off. And if they care about their reputation, they would be interested in signing off only on those students, who actually understand the topic.
So there will need to be exams. In the AI era, it is close to impossible to guarantee that a remote exam will accurately evaluate your understanding of a topic. So the exams would need to be carried out in person.
There is nothing children-specific in using such approach. It is merely one of the few still somewhat objective ways to actually test people's understanding of some topic.
Do you think 'At the end of the day, they can either demonstrate mastery of the material at an exam or not' is a reasonable expectation for an adult?
If so, how would an adult be treated while still expecting them to demonstrate mastery?
If not, what do you think a reasonable expectation for an adult should be?
You can learn remotely in his class. That's the whole point he is trying to make. Hybrid does not mean fully remote. If you want fully remote take a full remote class.
Are you after learning or an accredited degree?
Are you willing to pay to learn remotely from a university with crappy reputation?
Well that's part of the issue with for profit education, if the student pays significant money for tuition, is he a customer or a student?
"The student is the customer" is a stupid metaphor that only caught on because a lot of people would rather just bring in dollars than worry about "are we actually teaching anything?"
It's not a metaphor. They are literally the customer.
They're a customer in the same way that a martial arts student is a customer: getting your butt kicked is literally what you're paying for.
Sure. And just like with a martial arts student, if you kick their butt too hard, you will lose the customer and might get sued.
Sued for giving out hard work assignments? That doesn't sound plausible. Under what theory would a student have a case?
Denmark: "You must now verbally defend your essay." Students: sweating nervously "Uh, well, you see... the AI wrote it, but I agree with what it said?"
The more I read about articles like these, I grow even more convinced that the education industry as a whole, regardless of the country or stature, has kind of lost their plot.
Sometimes I even wonder if this is the outcome of sheer laziness, fear or both.
When I speak to professors, teachers and students - It's also surprising that "top tier" institutions are fighting AI harder (outside extremely specific courses like Harvard’s flagship CS50, MBA courses at Wharton (UPenn) and MIT) while "bottom tier" institutions are completely embracing it and rebuilding their curriculum around it. One CS professor at a "bottom tier" CSU mentioned to me that he's actively going completely "open book using AI" - students are allowed to use anything they want from Claude to Codex to OpenCode to finish assignments but the assignments have now changed from "blurt out quicksort" to "let's sort N natural numbers in a cache efficient way using least amount of resources". I would hire the latter over the former anytime. I am tired of interviewing candidates who can shit out quicksort before I can even finish my sentence but stare at me dumbfounded when I ask them to sort people's name serialized in unicode.
In my opinion, the bulk of traditional education has been a mix of memorization of facts, knowing inference rules, and applying inference rules to those facts, coupled with recall.
Before the age of LLMs, only very expensive-to-build rule-based expert systems were able to replicate that functionality. Humans were just simply cheaper and way more reliable.
In 2026, Frontier models are exceedingly, across the board, across industries, breaking records and challenging those notions on cost, capability and sophistication.
Trying to replicate how education used to operate pre-frontier LLMs is like forcing people to farm by hand in the age of automated tractors that have LIDAR, Vision, RTK on board.
I'm not discounting other elements of learning like collaborative debate, clinical/lab work, Socratic reasoning, emotional intelligence and the development of a professional network - but I argue these skills are not limited to a school or university setting. Infact, a lot of this is distorted in a school or university setting compared to the real world.
I have been filing my own taxes, including complexities like equity, real estate and business income for over a decade now, so it's not just a simple 1040 and 540. Reading through IRS documentation, talking to EAs and CPAs to fill in ambiguities and gaps was pretty expensive in terms of time and money.
Both Claude Opus and GPT 5.5 now, as of 2026, answer all my tax questions correctly. Those thousands of dollars in time and money I had spent has been replaced by a single $20 subscription. Unless tax codes drastically change every year, that $20 is a one time cost.
If I were to begin my tax journey in 2026, I would never had to spend those dollars: dozens of tax professionals are out of a job - all their education is for nothing.
It's entirely irrelevant whether they passed their exams by carving answers out on granite, taking their exams in a jail cell on an island proctored by Catholic Nuns under the watchful guard of automatic machine guns manned by T-1000s, I just don't need them anymore. For my usecase, these humans and their credentials provide 0 additional value over a one-time $20 expense, no matter how complicated it was for them to get their credential and what complex interpretive dance they had to do to impress the people awarding their grades.
All this "reject AI, do it by hand" is just insanity at worst and laziness at best.
That said, I'm personally unsure what the future of education in the age of tractors is like but removing weeds, planting seeds, watering them, all by hand is certainly not it.
It's my understanding that most countries - developed or developing - are bottlenecked on educating their masses due to lack of qualified teachers.
That's a bandwidth problem. Having those teachers listen to oral arguments from a handful of students is not solving the core bandwidth problem. All these shenanigans is doing a disservice to the public.
> In my opinion, the bulk of traditional education has been a mix of memorization of facts, knowing inference rules, and applying inference rules to those facts, coupled with recall.
These are important skills to develop, even in a world with LLMs.
> All this "reject AI, do it by hand" is just insanity at worst and laziness at best.
If a 3rd grader complained about having to learn multiplication because calculators exist, would you agree with them?
Why do we make kids learn how to do math if they can pull out their phone and open the calculator app? Because learning how to learn is important and there is value in understanding how the answer is produced, even if you have a machine that can produce it for you.
Universities can teach a separate class on how to use LLMs. The existing classes should be focused on teaching their existing subject matter, not becoming an extension of a how-to-LLM class
> Why do we make kids learn how to do math if they can pull out their phone and open the calculator app? Because learning how to learn is important and there is value in understanding how the answer is produced, even if you have a machine that can produce it for you.
I actually appreciate this question very much because this question is extremely pertinent to our discussion. Before I continue my response, I want to turn around and ask you:
1. What's your definition of "learn how to do math"?
i. Would it be sufficient if they proved they understood what addition, subtraction, multiplication and division was? Or do they need to be able to correctly calculate what 4592 * 314 is? What if they could show you using diagrams of squares and rectangles *why* (a + b)** 2 expands to the equation it does but refused to chart out results for various values of a and b?
ii. Would you fail a student who could reliably chart out results for (a + b)** 2 given various values of a and b, but failed to explain that using diagrams of squares and rectangles?
iii. Would you fail a student who could reliably add 1 + 3 or 7 - 5 or 7 * 5 but fail to compute 4592 * 314?
iv. Is a student who could reliably add 1 + 3 or 7 - 5 or 7 * 5 but fail to compute 4592 * 314 at the age of 5 superior or inferior to a student who is capable of doing the same at age 7?
2. If an exam is composed of tables of long division and your score on the test boils down to your ability to how many of those long divisions you can complete in 45 minutes, does the person who scores the highest have a superior understanding of division than the slowest student who suffers from mental fatigue due to an underlying, undiagnoised health condition?
3. When you say "understand how the answer is produced", what level of abstraction is acceptable to be not considered as cheating? Do they need to understand how the calculator physically computes the floating-point math, or is pushing the button enough? If pushing the button is cheating, why isn't using a base-10 shortcut algorithm also cheating? What if the button pusher explained to you precisely how IEEE 754 worked, how a digital calculator works and then refused to do long division citing it was a complete waste of their time and yours? Would you refuse to let them pass the class or fail them unless they yielded to your demands and complied with your specific definition of learning?
Is the math class also doubling as ability to pass compliance and behavioral standards or is it purely a test of mathematical ability?
4. If a student uses an LLM to generate the boilerplate code for a script, but can perfectly explain the architecture, debug the logic, and scale the deployment, have they failed to 'learn how to learn' just because they didn't manually type the syntax?
Educational resources and classroom time is a zero sum game. We cannot afford to educate everyone if they all require 1:1 coaching from a qualified human in a room that has limited space. Time in a day is zero sum as well. Every minute someone spends on doing the 135th long division is a minute they're not spending thinking if solving long division problems is a meaningful differentor in their long term success - whether it's the long division that will make them wealthy, happy and successful or something else entirely?
> i. Would it be sufficient if they proved they understood what addition, subtraction, multiplication and division was? Or do they need to be able to correctly calculate what 4592 * 314 is? What if they could show you using diagrams of squares and rectangles why (a + b)* 2 expands to the equation it does but refused to chart out results for various values of a and b?
I’m not an educator by profession but I’ve done a lot of training and mentoring.
There is no replacement for actually doing the work. People who study something but don’t go through the exercises feel they understand topics better than they do. It’s only when they are put in a position where they have to apply it that the cracks in their understanding are revealed.
This would 100% result in students who thought they could explain multiplication but only had a surface level idea. The knowledge would also be fleeting because actually doing the thing makes the knowledge stick more than just observing and understanding the thing.
So yes, you still need to have the students do the thing.
Replace math with writing and it will be more obvious. It’s easy to look at someone’s writing and critique it. It’s much harder to write well without practicing.
Depends on what skills you want the students to have.
In e.g., philosophy, allowing students to write their assessments with the help of LLMs changes the students’ depth of understanding, as well as their ability to synthesize, formalize, and drum up their own arguments and objections.
These are all useful skills, even in a post-LLM world.
> allowing students to write their assessments with the help of LLMs changes the students’ depth of understanding, as well as their ability to synthesize, formalize, and drum up their own arguments and objections
indeed but LLMs improve these capabilities. Let me explain:
Most educators behave as if using an LLM robs the student of their ability to think. That has not and never been my experience.
You can just tell an LLM to debate, find out inconsistencies and issues in your writing, and challenge you, and it will very well do that!
LLMs for me, cheaply replace teachers who can barely be accessed during office hours because they are so busy doing research or overwhelmed with grading coursework.
In a typical educational setup, I might have half an hour at best with a teacher who looks kindly upon me, and at worst, five minutes with a teacher who doesn't like being around me. An LLM has no emotion, but it has knowledge, it has inference rules, and it has the ability to push back, debate, and find out inconsistencies, bugs and gaps in my thinking.
Behavior that is exactly what I need consistently, in a good teacher.
A frontier LLM is a good teacher who doesn't play games, has no emotion, infinite patience and always available for $20/mo.
Two points:
There are two kinds of engineers in Finland. Some received a more theoretical education at research universities. Most went to more applied institutions that focus more on practical skills. Regardless of what employers say, they generally prefer the theoretical engineers from research universities. Because the higher status of those universities attracts more talented individuals, because theoretical understanding tends to stay valid longer than practical skills, and because it's easier to learn practical skills at work.
The kids who start their studies today are supposed to graduate in 2030, and they will probably retire around 2080. Focusing too much on the skills their early employers might want in the 2030s would be a huge misallocation of resources.
> because theoretical understanding tends to stay valid longer than practical skills, and because it's easier to learn practical skills at work
depends. If you're building a datacenter and need welders to weld two very specific set of metals together, allocating a PhD in metallurgy to do that welding, I argue would be a huge misallocation of resources.
Now your argument seems to be whether the PhD in metallurgy can pivot to doing something else entirely compared to someone who has been narrowly trained to only weld two very specific set of metals together.
If I interpret your take on the matter, your point seems to be that the former is "intellectually superior and more flexible" than the latter.
I dont take that position - I find most humans to be extremely capable. Just because one human was able to grasp the intricacies of metallurgy doesn't necessarily imply they have more capability than someone who didn't.
The fact that employers in Finland can be so very picky says more about the productivity of Finland than the employee's capabilities. At peak productivity, Finland would just not be hiring anyone who could breathe but import labor because there's not enough people in Finland.
I had two main points.
First, employers favor applicants from prestigious schools over those from less prestigious schools, because prestigious schools are more selective. That seems to be at least as true in the US as in Finland.
Second, theoretical education is more useful in the long term, because it's a better foundation for learning new skills. It can be a disadvantage when you are trying to get your first job, as you have fewer practical skills potential employers would need right now. Which is why, from an individual perspective, theoretical education only makes sense in higher-tier institutions, where the prestige can offset the initial disadvantage.
I heard your points - you communicated them very well. I failed to communicate that the first, while true, isn't going to matter in a few years time and the second is not true.
I will repeat what I said - if an employer arbitrarily restricts their talent pool to legacy credentials, they are optimizing for signaling rather than raw productivity.
By 2026, there is overwhelming anecdote and evidence that credentials and where you went to school have very little to do with demonstrated performance. (See citations 1-3)
Hence, it's mostly theatrics.
It seems like you're arguing that it's safer to attend prestigious schools because that increases your chances of being employed. I argue that in the age of LLMs, where the cost of learning is very low and speed of building prototypes (and hence learning and improvement) is rapid, it's easier than ever to start your own business.
> First, employers favor applicants from prestigious schools over those from less prestigious schools, because prestigious schools are more selective. That seems to be at least as true in the US as in Finland
This represents a trailing-indicator mindset typical of "MNC".
Yes, legacy corporations still filter by prestige because they want to signal to others they hire only the best, but that's because in extremely large orgs like theirs, it's impossible for a single person's contributions (outside the C-Suite) to have an outsized effect on that org, so they resort to proxy metrics, like credentails, for signalling.
In real life though, credentials have an extremely weak correlation with impact (see citations).
What this translates for an end user, is that the end user doesn't care about the credentials of the person who built the product. I explain that below.
On the frontiers of excellence - startup founders will tell you that hiring people from prestigious schools is a liability over those who might only have a high school degree because the prestigious school graduate is entitled and is looking to build a resume so they can jump ship in 2 years instead of the barely high school degree student who's hungry to prove they are worth investing in.
The selectivity of prestigious schools is gamed heavily in the U.S where you have literally floors of "prep schools" full of trainers telling you which piano lessons to take, which tennis tournaments to play at, which charities to volunteer for and which SAT question types to learn well. Getting into Harvard is less about ability and more about checking off boxes on a checklist.
Notice that absolutely none of this actually has to do with what the student actually wants to do in life and everything to do with impressing the gatekeepers to admissions.
However, make no mistake, the end customer rarely cares about the credentials.
When you watch Netflix or play the XBox, I bet you don't look up the credentials of the production team or the engineers who built it - you enjoy the content and vote with your dollars - and that's real life.
HN is full of people who don't have anything past a high-school degree but they were obsessed with solving one problem well and now they are multi-millionaires and billionaires because they ended up having an outsized impact. There are far more in that bucket who are comfortable but not multi-millionaires (so you never hear about them).
The money is not the end result but proof of impact and this proves my point.
Let's even discard that, citing survivorship bias. Let's reduce my argument to one fact: engineering/code doesn't care about the author's pedigree - all that matters is whether the system compiles, works correctly and scales under load and a prestigious university certainly doesn't have exclusive rights to those skills.
> theoretical education only makes sense in higher-tier institutions, where the prestige can offset the initial disadvantage
That doesn't make any sense. An institution will hire as per their needs: higher-tier, lower-tier, mid-tier all have their unique needs.
The reason why higher-tier institutions can afford to hire advanced degree graduates is because lower-tier, mid-tier simply cannot afford that baggage: highly credentialed individuals often won't work on tasks they consider "beneath them", the actual "plumbing" of engineering, which makes them useless to lean, high-velocity teams - which is precisely the point the founders at start ups make.
I think you're conflating multiple issues together altough you're not alone in the population that strongly advocates for education as a proxy for ability. They have nothing to do with each other and have a lose correlation.
Here's a thought experiment:
- Kid A (Rural Finland): Has internet access, a Claude Code subscription, and a singular obsession. They spend their time learning how to write a provably correct, memory-safe OS kernel. Through sheer, unhindered iteration and thousands of failures, they master thread management, hardware concurrency, durable scheduling, and modern CPU architecture.
- Kid B (New York City): Attends a prestigious prep school. Their entire mental bandwidth is consumed by a generalized conveyor belt: memorizing 100 years of Eupropean and U.S. history, computing the molality of an acid drop, identifying stereocenters in organic chemistry, and writing essays on what Socrates said to Glaucon. They barely touch operating systems because there is no time left after juggling DBMS trivia, analog design, and quantum physics to satisfy a rubric.
Which person would you listen to or hire when it comes to building an OS?
Now your argument might then pivot to "but what if instead of building an OS, I wanted someone to do algoritmic trading on Wall Street? Ha! I got you there! Kid A knows nothing about that, they only know how to write a kernel!"
Kid A doesn't care. Their singular obsession is OSes and they are world class at it. Their brains aren't clouded by molality and stereocenters - an OS kernel certainly doesn't care about them. If in the future, we had a CPU that worked off organic chemistry, maybe he will learn about it or maybe not and there will be Kid C in the unlikely case Kid A failed to build a trading engine from scratch, fail 1000 times, and learn the math on the fly just like they did for the kernel.
---
[1] Google's Internal Data (Laszlo Bock): In 2013, Google's SVP of People Operations publicly revealed that GPAs, test scores, and university prestige had zero correlation with employee success. They explicitly began hiring more candidates with no college degrees because work-sample tests were far better predictors of impact.
[2] The Dale & Krueger Study (NBER): A famous National Bureau of Economic Research study found that students who were accepted to Ivy League schools but chose to attend less prestigious state schools earned just as much money later in life. It proved that the ambition of the student drives success, not the prestige of the institution.
[3] The Schmidt & Hunter Meta-Analysis: The gold standard in industrial psychology (examining 85 years of hiring data) found that "years of education" has a dismal 0.10 correlation with job performance, whereas "work sample tests" and "cognitive ability" are highly predictive.
The actual Finnish kid who wrote a somewhat successful OS kernel was born in Helsinki, to an upper middle class family with a communist father. He chose to study in the most prestigious university in the country, largely due to his family ties. Because the CS program was theoretical, it didn't take that much of his time. He could enjoy student life and also pursue his other interests. The OS kernel emerged from the latter.
That's what education in a research university is like, when you actually take advantage of it. You don't have to focus on whatever seems useful and practical in the short term, you don't have to compete against your peers, and you don't have to overwork yourself. Instead, you have a socially acceptable opportunity to study whatever interests you, and you hopefully gain the attitude that you can learn whatever you need to learn.
This is the way
Denmark is a very small country and what works there doesn’t necessarily work at the scale of the US or China.
Would it work in Wisconsin, Minnesota, South Carolina, Alabama, Louisiana, Kentucky, Oregon, Oklahoma, Connecticut, Utah, Nevada, Iowa, Puerto Rico, Arkensas, Kansas, Mississippi, New Mexico, Idaho, Nebraska, West Virginia, Hawaii, New Hampshire, Maine, Montana, Rhode Island, Delaware, South Dakota, North Dakota, Alaska, DC, Vermont or Wyoming?
They all have a lower population than Denmark.
All but six states (CA, TX, FL, NY, PA, IL) have less than double the population of Denmark.
Top US universities actually have higher teacher/student ratios than top EU schools, and I wouldn't be surprised if this pattern holds in general, so it would be easier to do in the US than it is here.
This is about Danish high school exams.
Oh, in that case it is actually flipped.
Piling on. Why? Are class sizes that different? US and China have many more professors too, is it not proportional?
The US consists of many small parts too, and it may work in each of those small parts.
The important variable is the staff/student ratio, not the size of the country. With few students per class, or many teachers, this scales fine. I'm running similar exams for my (UK) Masters modules, but colleagues are reluctant to do it for the enormous compulsory survey courses as there are several hundred students in them (vs up to about 50 in the MA ones).
For a large compulsory survey course, a well written multiple choice test should suffice. If the test is well constructed, teaching to the test should cover exactly the required material and be no impediment.
Why not? This is such a thought terminating statement. No actual reason.
Tell me you're not a programmer w/o telling me. What works at scale can be fundamentally different than what works in a small startup (Denmark).
Could you point out a reason why education, which can be standardized at least to a state scale¹, cannot be standardized to the state scale utilizing oral defenses? Also, can be ≠ is. Not everything is a startup lol
¹https://www.doe.mass.edu/frameworks/
You said it can be fundamentally different, and of course it can. But “it doesn’t scale” isn’t an argument by itself. In many areas, scale actually creates efficiencies. You’d need to explain what specifically about the larger scale makes the Danish approach unworkable in the U.S
If in person exams are seemingly impossible in US due to scale, we have to conclude that before internet US had no education?
For parallelizable workloads it scales just fine, and giving exams is perfectly parallelizable. Any good programmer would know that.
This is a nightmare for students that don't think on their feet.
Can you explain why? As I understand it, you're supposed to be able to talk about the work in which you already invested the mental energy to write. Writing involves much more "brain work" than speaking, so if you've truly written the work in question, you should be capable of speaking about it -- unless you have some sort of disability, in which case, there are probably alternative avenues.
This is definitely something you can't force onto students on a whim. This is a something that can be taught and worked up to though.
Part of college is growing as a human, and part of that is social interaction.
For the same reason a lot of universities in Europe require you to take philosophy. It's completely unrelated to C++ or data-structures, but it adds to your acumen as a whole.
Not that different from some universities that require you to be able to swim[0].
[0]: https://perec.columbia.edu/swim-test-requirement
I only know the Uni system of a few European countries, but generally I would expect Philosophy to be a prerequirement that you take for your entrance examinations, not during studies per se (unless you're studying humanities, of course).
Yeah as much as I like the idea, I know there will be some brilliant and driven students who would have a hard time with this due to temperament. For me, if I’m excited about something, I can talk for hours. Maybe not so for everyone though. Then again, not all brilliant and driven students test well either, so… pick your poison?
If you don't have enough of a mastery over a subject to rapidly respond, maybe you are not fit to pass exam certifying your mastery over a subject.
I know world-leading experts in their field that avoid oral debates precisely because they don't think on their feet.
Millions of students are now donating to neuralink so they can have stealth LLM answers on the fly. Teachers will have to count the number of load bearing per paragraph.