NYU Professor Uses AI Oral Exams to Fight Academic Cheating at a Cost of Just 42 Cents

On addressing the issue of academic cheating brought by artificial intelligence, a professor at New York University adopted an innovative approach. In a course titled "AI/ML Product Management," he conducted an oral exam experiment, which revealed gaps in students' knowledge and shortcomings in his own teaching. Professors Panos Ipeirotis and co-instructor Konstantinos Rizakos found that many students who submitted high-quality written assignments were unable to clearly explain the basic decisions in their work during the oral exam.

Image source note: The image is AI-generated, and the image licensing service provider is Midjourney

To address the limitations of traditional assessment methods, they introduced an AI oral exam, utilizing a voice AI agent based on ElevenLabs. The exam consisted of two parts: first, the AI agent asked students questions about the goals, data, modeling decisions, evaluation, and failure patterns of their final project; then, the AI randomly selected a case from the course for further questioning. 36 students took the oral exams within nine days, with each exam lasting an average of 25 minutes, and the total cost was only $15, costing just 42 cents per student.

Although the first version of the AI oral exam had some issues, such as students finding the AI's tone too strict and the AI sometimes asking multiple questions at once, leading to communication problems. After adjustments, the AI's performance improved. In terms of scoring, they used three AI models—Claude, Gemini, and ChatGPT—to score the exams. The initial consistency of the scores was low, but after mutual review, the accuracy of the scoring improved.

Although most students found the AI oral exam more stressful than written exams, 70% of students said that this type of exam indeed tested their actual understanding. Professor Ipeirotis pointed out that AI technology made the implementation of oral exams feasible, and students can improve their abilities by practicing interactions with AI.

Key points:

📊 Students performed poorly in the oral exam, revealing the gap between knowledge and written assignments.  

💰 Using AI oral exams costs only 42 cents per student, greatly reducing the economic burden of traditional oral exams.  

🤖 Students found the AI oral exam more stressful, but most believed this method better tested their actual understanding.