Mollicks And Penn Release Report On The State Of Prompting
“This is the first of a series of short reports that seek to help business, education, and policy leaders understand the technical details of working with AI through rigorous testing. In this report, we demonstrate two things: There is no single standard for measuring whether a Large Language Model (LLM) passes a benchmark, and that […]
We Should Call Them “Mirages” Not “Hallucinations”
“But a metaphor of hallucination reinforces the misconception that AI is conscious; it implies that AI experiences reality and sometimes becomes delirious… Ultimately, we chose a more fitting term: AI mirage. Just as a desert mirage is an artifact of physical conditions, an AI mirage is an artifact of how systems process training data and […]
Early Research: How To Effectively Use GenAI For Planning
“In fall 2023, all teachers were novice users or had never tried generative AI. By spring 2024, the teachers separate into three distinct groups: (1) those who seek generative AI input (i.e., thoughts or ideas about learning plans) and output (i.e., quizzes, worksheets), (2) those who only seek generative AI outputs, and (3) those not […]
What Happens When AI Tutoring Gives The Answer, Becomes A Crutch
“Consistent with prior work, our results show that access to GPT-4 significantly improves performance (48% improvement for GPT Base and 127% for GPT Tutor). However, we additionally find that when access is subsequently taken away, students actually perform worse than those who never had access (17% reduction for GPT Base). That is, access to GPT-4 […]