5 minEconomy
Librarian Proposes Four-Type Framework for Judging AI Answers
A University of Virginia librarian argues that accuracy alone is insufficient for evaluating AI-generated responses, proposing a typology of factual, interpretive, constructive, and strategic answers to help users assess reliability.
As AI-generated answers increasingly replace traditional search results, users need more than a simple accuracy check to judge whether a response is trustworthy, according to a University of Virginia librarian who has proposed a new framework for AI literacy.
The librarian, who also leads national efforts to develop AI competencies for library professionals, argues that AI answers fall into four broad categories — factual, interpretive, constructive, and strategic — each requiring a different kind of evaluation. The typology was first published in The Journal of Academic Librarianship.
«Debate about AI answers has focused on accuracy: Did the system get the answer right?» the librarian wrote. «That matters, but accuracy is only one test. Each kind of answer requires a user to judge something different.»
A factual answer makes a claim that can be checked against evidence, such as the founding date of a university or the chemical symbol for gold. To assess it, users should verify the claim against an appropriate source and follow any cited links rather than treating the AI response itself as proof.
An interpretive answer is built on evidence but does not have a single takeaway. Questions like how much screen time is too much for a teenager or whether remote work raises productivity depend on which evidence is included, what is left out, and how disagreement is understood. The American Academy of Pediatrics says there is no exact recommended amount of screen time for teens and emphasizes the kind of use and what activities it might displace. To evaluate interpretive answers, users should ask what evidence the system emphasized, what it omitted, and whether another defensible interpretation exists.
Constructive answers are made rather than discovered. When AI is asked to draft a cover letter, write a eulogy, or suggest a lesson plan, there is no single correct result. These responses can be judged by purpose, audience, and voice. A eulogy can be grammatically perfect and still sound nothing like the person delivering it, or it may land flat on family members hearing it.
Strategic questions ask what to do — whether to take a daily aspirin or buy a house. The answers combine information with judgment about goals, risks, trade-offs, and personal circumstances. In one example, a search about daily aspirin returned warnings about risks, advised consulting a medical professional, and offered more tailored information if age, cardiovascular history, and bleeding risk were provided. That caution matches U.S. Preventive Services Task Force guidance, which says the decision to start low-dose aspirin for prevention of heart attacks and strokes should be individualized and weigh cardiovascular benefit against bleeding risk.
For strategic answers, users should ask what the system would need to know before its advice could reasonably apply to them individually, consider the stakes and alternatives, and decide whether a qualified person should be involved. A useful follow-up question is: «What details about my age, medical history, or bleeding risk could change this advice? What should I discuss with my doctor before deciding?»
The four categories are not airtight boxes. A single AI response can reflect several types, and the type of answer can shift within one reply without a noticeable change in voice. Reporting what a medical guideline says is different from deciding how it applies to a particular person, yet AI presents all four types in much the same fluent, authoritative form.
The librarian’s questions began as ordinary Google searches rather than a chatbot. The AI-generated responses arrived with links appended, offering context, acknowledging complications, and providing tailored guidance when additional information was supplied. The responses were useful, but the final judgment remains with the user, who has to live with the outcome.
The framework aims to help users recognize what kind of intellectual work the AI agent did for a response and whether that reply is ready to use or needs more investigation. As AI-generated answers become more common, the librarian argues, the first question after receiving an answer should be not only whether it is accurate but what kind of answer it is.
