What an exam, an internship, and the question “what if?” reveal about the limits of a correct answer.
Imagine two engineering graduates on their first day at a factory.
Both passed the same university courses. Both know the equations describing the machines around them. One was particularly good at examinations: give her a well-defined problem and the relevant figures, and she could almost always find the right answer.
But the factory has no examination paper waiting on the workbench. A machine has stopped unexpectedly. The maintenance log is incomplete, the noise sounds different from yesterday, and production needs to restart. No one has told the graduates which chapter of the textbook applies.
One graduate searches for the appropriate formula. The other first asks what is actually wrong with the machine.
Here is the difficulty that examinations can hide: knowing how to solve a problem is not the same as knowing which problem needs solving—or when a familiar solution applies.
The question before the question
Consider a standard physics exercise. It tells you the mass, the distance, perhaps the coefficient of friction. You may not know the answer, but the problem has been framed for you. The relevant conditions have already been selected; your job is to work within them.
At work, you may first have to discover whether friction matters at all. Perhaps the motor is overheating. Perhaps a sensor is reporting the wrong value. Perhaps two parts that work separately interfere when assembled. Perhaps the apparent fault is only a symptom of an earlier decision.
A classroom problem often gives you its boundary. A real job may require you to find that boundary before you can solve the problem inside it.
Even the goal may be less obvious than it first appears. Restoring production immediately might raise a safety risk. A quick repair might shorten the machine's life. The person doing the work may need to discover not only the cause of the fault but which consequences matter.
This is not a claim that schools never teach open-ended problems, or that work never follows a fixed procedure. The difference is one of emphasis. When a task has been carefully framed in advance, success shows that someone can perform under those conditions. It does not settle what will happen when those conditions have to be discovered, questioned, or changed.
The internship: when the world changes the conditions
This is one reason internships, apprenticeships, and supervised practical work can matter.
A student who encounters real equipment discovers details that a textbook quite reasonably leaves out: imperfect measurements, delayed information, unusual failures, the need to coordinate with other people, and constraints imposed by time, cost, and safety.
More importantly, reality answers back. A student expects a repair to work, but the machine behaves differently. A measurement does not fit the initial explanation. A senior technician asks a question the student never thought to ask. Feedback reveals where an apparently correct explanation stops being sufficient.
Practical experience has value not merely because it is practical, but because it confronts the learner with conditions that were not chosen to make an exercise tidy.
Of course, repeating the same routine for months does not necessarily prepare someone for unfamiliar failures. The value of a placement depends partly on which situations the student encounters and what they learn from them.
The student who keeps asking “what if?”
But real-world placements are not the only way to go beyond the example in a textbook.
A teacher explains how a machine behaves under standard conditions. One student learns the method, solves the assigned problem, and moves on.
Another student stays with the problem, perhaps only in her mind.
What if the load doubles? What if the sensor is wrong? What if the temperature rises? What if an assumption behind the equation no longer holds? Which part of the conclusion changes first?
No teacher required these extra questions. Yet the student has begun to explore conditions beyond those supplied in the lesson.
This is one ordinary meaning of learning to generalize from an example. Instead of storing only the answer, the student is asking why the answer works, and under what circumstances it would no longer work.
When that student later encounters an unfamiliar machine, she may seem to understand it unusually quickly. Perhaps she has already considered similar possibilities. Perhaps she is now checking which of her earlier assumptions fit what she sees. The speed is real, but it does not mean she was born knowing how that particular machine works.
Imagining different conditions cannot replace observing the real world. A thought experiment may preserve assumptions that turn out to be false. But it can prepare useful questions before the learner has to face the consequences of a real decision.
The experienced worker: when yesterday's conditions return—or do not
Experience offers a third route. A technician who has maintained a machine for twenty years may recognize a routine fault almost instantly. Years of varied observations, repairs, and outcomes have made some patterns familiar.
That experience is valuable precisely because similar conditions recur.
Now replace the machine with a redesigned version. A familiar sound may have a different cause. A repair that worked on the older model may be unsafe or ineffective on the new one.
The experienced technician may adapt faster than anyone else. Or familiarity may tempt them to overlook the change.
Internships expose learners to conditions chosen by the world. Asking “what if?” lets them explore possible changes in their own minds. Experience builds a history of situations that actually occurred. Each can help, and each can mislead, if the learner mistakes yesterday's conditions for today's.
So the question is not simply how much a person knows or how long they have worked. It is how far that knowledge remains applicable when the conditions differ from those under which it was learned.
What learning research tells us
Researchers have studied this problem for decades under the name transfer of learning.
The National Research Council's How People Learn discusses why remembering a taught solution differs from applying knowledge across situations, and why contrasting cases and questions about changed conditions can help learners recognize important principles. See How People Learn, Chapter 3 (2000).
Perkins and Salomon distinguish familiar situations that readily trigger practiced responses from situations that call for deliberate abstraction and a search for connections. See “Transfer of Learning,” in International Encyclopedia of Education, second edition (1992).
Barnett and Ceci examine the many ways two situations may differ—in the task, setting, timing, and other dimensions—and caution against speaking of “far transfer” as though it were a single, simple quantity. See Psychological Bulletin 128(4), 612–637 (2002).
The research does not make school learning unimportant. Structured teaching gives learners concepts that would often be costly, slow, or unsafe to discover from scratch. But the score achieved under one set of conditions is not a complete description of what the learner can do under another.
What if an examination allowed the student to question its answers?
This suggests a small educational experiment that interested teachers could try. We propose it as a possibility, not as a method whose benefits have already been demonstrated.
I used to answer some multiple-choice questions by substituting the offered answers back into the problem. Sometimes testing the choices was quicker than working out the answer from the beginning. That is not necessarily bad reasoning: checking a candidate solution can be an efficient and legitimate strategy.
But it reveals something about the format. The question has already supplied a set of possible answers, and students can often work backward from those answers without having to establish whether the set is complete.
Consider keeping the usual four choices—A, B, C, and D—and adding a fifth:
E. None of the available choices can be justified. Explain your answer.
A student who selects E would also write what they believe the answer should be, if they can identify one. Alternatively, they might explain why the information given is insufficient to choose an answer at all. These are different situations: the correct answer is missing is not the same as the evidence is insufficient to determine which answer is correct.
Some questions would still have an ordinary correct answer among A–D. In others, the intended response might be E because a different answer is needed. In still others, the problem might genuinely lack enough information for a definite answer. A teacher could also replace one of the four original choices with this option rather than adding a fifth.
The point would not be to trick students by making every question uncertain. It would be to let them occasionally ask something that the standard format usually takes for granted: Why should I assume that the correct answer must already be among these choices?
Would such questions improve students' judgment, or merely change their test-taking strategies? We do not know. That is why this is a suggestion to experiment with, not a claim of established educational effectiveness. Teachers who are interested could compare it with ordinary multiple-choice questions and see what students actually learn.
What should a good examination measure?
An examination can reasonably ask whether students understand material they have been taught. There is real value in learning definitions, principles, methods, and calculations accurately.
But it can also ask whether students notice a missing fact, an invalid assumption, or a problem that has been framed too narrowly.
A strong education need not choose between knowledge and independent judgment. It can teach the established answer and teach students to ask why that answer is appropriate here.
The same applies when we evaluate a new employee. A strong first performance may reflect relevant prior knowledge, carefully imagined alternatives, rapid observation, or several of these together. An uncertain first day may reflect unfamiliar equipment rather than weak understanding. We cannot reliably reconstruct how someone's competence arose from the final answer alone.
What matters is not simply whether the person can produce a correct response to a familiar question, but whether they can recognize when the question—or the conditions on which the answer depends—has changed.
Beyond the classroom
A similar issue appears in artificial intelligence.
A benchmark, like many examination questions, evaluates a system under specified conditions. Real deployment may involve missing information, shifting assumptions, or a problem that has not yet been framed correctly. Performing well on the benchmark does not by itself show that the system will recognize those changes in the world.
Before asking only how many answers a student—or an AI system—has learned, we might ask a prior question: Does it know when the answers it has been given are no longer enough?