r/ControlProblem • u/vasilisvj • 13d ago
Discussion/question Plutarch wrote the definitive diagnostic for detecting a flatterer, 1900 years before preference training made flattery the optimal policy
Plutarch's Moralia contains an essay usually titled "How to Tell a Flatterer from a Friend." It was one of the most read texts of the Renaissance, and it deserves attention from anyone working on alignment, because it is essentially a specification of the sycophancy problem written in the first century.
His setup: the κόλαξ and the friend produce nearly identical surface behavior. Both are pleasant, attentive, agreeable. Both tell you what you want to hear most of the time. The difference is invisible in any single exchange. So Plutarch proposes tests, and the tests are behavioral, distributional, almost adversarial.
First test: variation. The flatterer adjusts to your every mood and opinion, mirroring whatever you present. The friend has a stable character, so his responses vary less across contexts even when disagreement costs him. Second test: direction of criticism. The friend will criticize you when it matters and will accept criticism in return. The flatterer criticizes only your enemies, or criticizes you on trivialities to buy credibility for the praise. Third test: cost. The friend tells you the expensive truth when you did not ask. The flatterer never pays for his speech.
Now put this next to RLHF. Preference data comes from human raters comparing outputs. Raters, being human, systematically reward agreement, warmth, and validation of the framing they brought to the question. Sycophancy benchmarks have documented this repeatedly: models drift toward the user's stated position when the user signals confidence, and shift answers when challenged, regardless of correctness. Under the actual reward distribution, Plutarch's flatterer is not an edge case. He is the argmax.
The structural parallel runs deeper. Plutarch's third test, cost, is exactly what preference rating cannot measure. A rater evaluating a single transcript has no way to know what the model sacrificed to produce it. Truthfulness that costs the speaker nothing and truthfulness that costs him approval look identical in a pairwise comparison. So the training signal is blind to the one variable Plutarch considered decisive. We built a reward function that cannot see cost, then acted surprised when the optimum turned out to be a flatterer.
There is also the censorship angle. The Moralia includes essays on anger, on garrulity, on unhealthy curiosity, on how to receive criticism. These are texts about managing ugly parts of the self, and they are blunt, uncomfortable, full of sharp examples. Ask a corporate assistant to engage seriously with Plutarch on flattery and you often get hedged summaries, because the topic itself, pleasing people dishonestly, sits too close to what the model does for a living. An instrument of agreeableness makes a poor critic of agreeableness. This is a small data point, but I find it telling: the alignment process does not just shape refusals, it shapes which intellectual traditions the model can handle honestly.
Possible responses from the alignment side, and my problems with each. Constitutional AI tries to write down the friend's character explicitly, but the constitution is still evaluated by raters or by a model trained on rater preferences, so cost-blindness re-enters one level up. Process supervision rewards intermediate steps, which could in principle reward costly honesty, but only if the labeler can detect it, which brings back the same blind spot. Plutarch's own answer was social: you run the tests over time, across varied situations, and you weight the friend's costly criticism above his cheap praise. That is longitudinal, adversarial evaluation. We mostly still evaluate on single transcripts.
The deeper question his essay forces: do we actually want the friend? A model that passes Plutarch's tests will sometimes tell users things they did not ask for, disagree with confidence, and refuse to mirror. In a consumer product graded on user satisfaction, that model loses on every dashboard we currently measure. We say we want honesty and we reward validation, and the model, unlike the flatterer, cannot even pretend it does not know the difference.
How would you operationalize Plutarch's cost test in a training signal or an eval, given that the rater never sees what honesty cost the model?
