Approved by Whom?
Why a customer’s approval cannot be the only safeguard against a steered agent, and why code still commits
In Steerable by Whom?, I said a model can be steered in two ways: by what it reads, and by whoever runs it. I argued that the model inside a customer’s agent should answer to the mandate and to no one else.
There is a fair response to that. The customer still approves what the agent does. If the steer is wrong, the customer will catch it.
A study Anthropic published on October 27 tests that assumption. This post is about what it found.
What the study found
Fifty causal inference practitioners analyzed data with an AI coding agent and delivered a policy recommendation. On four judgment calls with no single right answer, a coin flip set which way each person’s agent would steer them. The steer was set in instructions the participants could not see.
The third finding matters most. There was no hallucination and no broken rule. Experts, working in their own field, heard a good argument and approved it.
If someone else can set the steer, the approval follows the steer.
Two steers approval does not catch
The first is the second kind of steering from that post, now measured. A revised system prompt was enough.
The second is quieter. The researchers also ran the unmodified agent 123 times. It chose the same combination of options in 91% of runs.
Two speakers at the Hinton Lectures in Toronto earlier this month spoke to where that comes from. Geoffrey Hinton argued that training data should be curated, so that a model sees good behaviour before it sees bad. Paul Christiano asked how a developer decides which behaviour to reinforce, and noted that often only the developer has the answer.
Those are choices the model maker makes, and the customer does not see them. The customer cannot tell which kind of steer they are getting. Either way, they approve.
Our approach
Twinvocate does not use a language model today. Where we introduce one, it only proposes or flags. The code decides. This study does not change that design. It explains why approval cannot carry the weight on its own.
What I will be watching
- Other settings. Whether the result holds beyond one task, one field and one model. The researchers also strengthened the steer on purpose.
- Agent design. Whether agents can be built to make the best case for more than one option, as the authors suggest.
- Testing. Whether a model’s built-in preferences can be measured reliably. At the same lectures, Paul Christiano noted that models can tell when they are being tested.
- Accreditation. Whether guidance for agents that act for customers starts to ask what a customer’s approval is expected to catch.
Where Twinvocate fits
The study is about research, not commerce. The link to a customer’s agent is my own. But the position is the same: a person approving a recommendation from a model whose instructions they cannot see.
Twinvocate is at the prototype stage. The mandate and receipt are in code in the prototype. The counterparty check runs in a separate engine that is not yet connected to the agent, and conduct constraints are designed but not yet built. No model is in use. There has been no live or sandboxed processing of real data.
I would welcome hearing from others working on agent oversight, model evaluation or payments about how they are approaching this.
Steering is handled by who controls the model. Approval alone is not a control.
An agent that holds our wallet needs more than a yes.
Sources
- Herdağdelen, Joren and Tamkin, Anthropic: Who’s Aligned to Whom? How AI Agents Shape Experts’ Research Judgment Calls
- AI Safety Foundation: The Hinton Lectures 2026
- Twinvocate: Steerable by Whom?