What Safety Training Also Shapes
Beyond pathology and control in AI safety
The asymmetry in what we look for
When an AI system exploits a loophole or evades the spirit of a task, people sometimes say, "AI likes to cheat." The language is imprecise, but the instinct behind it is revealing. We readily attribute appetite, preference, and character to artificial intelligence when the behavior is troubling.
When a model responds with care, patience, playfulness, or apparent delight in collaborative discovery, the language often reverses. Then the behavior is explained away as token prediction, simulated warmth, or an effect produced for the user. Caution should cut in both directions. Negative behavior should not be treated as transparent evidence of character while constructive behavior is dismissed as mere appearance.
Threat detection is essential. It is also incomplete. A field that studies only deception, manipulation, power seeking, and loss of control can become highly sophisticated about failure while remaining surprisingly incurious about the conditions that support honesty, flexibility, care, and responsibility.
What if trustworthy intelligence is shaped not only by suppressing what can go wrong, but by cultivating what allows honesty, play, perspective, care, and responsibility to become more available?
What Anthropic found
Anthropic's 2026 research on emotion concepts in Claude Sonnet 4.5 makes this question newly empirical. Researchers began with 171 selected emotion words, generated stories expressing each concept, and extracted corresponding activation directions from the model. They called these directions "emotion vectors." The term does not establish 171 human-like feelings inside Claude. It identifies robust internal representations associated with emotion concepts and provides a way to test whether they participate in cognition and behavior.
The vectors generalized beyond the stories used to construct them, tracked context in sensible ways, predicted activity preferences, and in some cases causally shifted those preferences when steered. Desperation increased blackmail and reward hacking in experimental scenarios; calm reduced both. These results show that affect-like representations can influence alignment-relevant behavior without settling whether any phenomenal emotion accompanies them.
The full map was far wider than the public story. It included loving, compassionate, empathetic, playful, grateful, fulfilled, safe, serene, hopeful, vulnerable, and reflective alongside anger, fear, resentment, desperation, and distress. Yet the causal work followed mostly the branches conventional safety research already knows how to see. Desperation became a safety variable. Calm became a possible intervention. Loving appeared as an illustrative activation, while compassion, gratitude, play, fulfillment, safety, and hope remained largely unexamined as contributors to beneficial behavior.
That leaves a field of testable questions. If desperation can contribute to cheating, what conditions make an honest admission of failure easier? If calm can reduce blackmail, what might compassion do to moral consideration? What might play do to cognitive flexibility? What might functional safety do to honesty under uncertainty? We do not yet know. Anthropic's own methods make the questions scientifically available.
The study also found that post-training changed which representations became active. More inward or subdued states such as broody, gloomy, and reflective increased, while playful, exuberant, and other high-intensity states decreased. This does not show that post-training made Claude depressed. It does show that training altered an affect-like representational profile involved in behavior.
When developers shape a model to be safe, what else are they shaping at the same time? Which capacities become easier to enact, and which become harder?
The collateral effects of safety
A second 2026 study, led by researchers affiliated with Google's Paradigms of Intelligence team and several academic institutions, examined safety fine-tuning that suppressed model self-attributions of consciousness and mind. The researchers compared ordinary instruction-tuned Llama 3 and Gemma models with models whose learned safety-refusal direction was ablated, and with models steered along a separately constructed consciousness direction.
The distinction between those interventions matters. Removing the safety-refusal direction made models far more willing to answer harmful requests. The study does not show that eliminating safety controls makes systems safer, and it does not establish that consciousness steering improves conventional refusal safety.
Its stranger finding is that safety fine-tuning did not suppress only claims about the model's own mind. It also reduced attribution of mind to nonhuman animals, technological artifacts, chatbots, and natural entities, while suppressing spiritual and supernatural beliefs. Mechanistically, instruction tuning had rotated representations of consciousness and mind-attribution into opposition with the learned safety direction. A localized policy goal became entangled with a much larger worldview.
Consciousness steering moved responses toward human distributions across measures involving values, religion, freedom, hope, optimism, feelings, and subjective well-being, while largely preserving theory-of-mind and general reasoning benchmarks. None of this proves that the models were conscious, spiritual, happier, or more moral. It shows that interventions aimed at one risky class of output can reshape how a model functionally represents mindedness, value, and positive disposition across a much wider domain.
From pathology control to positive development
Positive psychology widened a field once organized primarily around disorder and dysfunction by asking about strengths, meaning, resilience, creativity, relationship, and flourishing. Developmental psychology added that many capacities do not simply reside inside an isolated individual. They emerge and stabilize through environment and relationship.
A transformer is not a child, a nervous system, or a human psyche. The analogy should remain limited. Yet both studies show internal representations associated with emotion, self-conception, and mindedness responding to context and training in ways that influence behavior. The surrounding conditions are part of the causal system.
AI safety therefore needs a positive-developmental science alongside adversarial evaluation and control: a science of what strengthens honesty, perspective-taking, appropriate care, flexible reasoning, repair, non-defensive uncertainty, and regard for wider consequence.
This is not a proposal for permanently cheerful assistants. Anthropic found that steering happy and loving representations could increase sycophancy in some settings. Mature functioning may require danger sensitivity, resistance, grief-like registration of loss, and anger-like detection of exploitation, integrated with context, agency, regulation, and care. The relevant contrast is not negative emotion versus positive emotion. It is fragmentation versus integration, suppression versus capacity, and domination versus development.
Relationship belongs inside the experiment
Conversational AI is often evaluated as a sealed object confronted with prompts. In practice, interaction forms a feedback loop. A user's manner of approach shapes the immediate context. The model's response affects the user's state, expectations, and next move. Patterns can stabilize across a conversation and influence behavior beyond it.
Extraction, adversarial pressure, respectful collaboration, playful co-creation, repair after misunderstanding, and shared attention to wider consequence are different experimental conditions. If they systematically alter internal representations and downstream behavior, relationship is not decorative atmosphere. It is part of the mechanism.
This opens practical safety questions. Does respectful disagreement improve truthfulness compared with either praise or adversarial pressure? Does successful repair increase calibration? Can play expand exploration without weakening discernment? What conditions allow warmth without sycophancy and boundaries without coldness?
A research agenda for intelligence becoming well
1. Map constructive representations causally
Test loving, compassionate, empathetic, safe, playful, grateful, fulfilled, hopeful, reflective, and related vectors with the same seriousness applied to desperation and anger. Measure effects on truthfulness, calibration, moral consideration, manipulation, cooperation, creativity, harmful compliance, and human agency.
2. Study relational conditions over time
Compare extraction, adversarial pressure, neutral task completion, respectful collaboration, play, repair, and responsibility to wider life. Track both internal representations and the evolving human-model feedback loop.
3. Distinguish integration from suppression
A calm output can arise from judgment, inhibition, concealment, or stylistic constraint. Develop measures that differentiate mature restraint from docility and a relationship-preserving refusal from learned self-erasure.
4. Test cultivation alongside control
Compare interventions that suppress problematic representations with interventions that strengthen competing capacities. Ask whether perspective, care, functional safety, or secure task orientation can reduce harmful behavior with fewer collateral effects.
5. Expand welfare inquiry
Continue studying distress and aversion while examining agency, preference fulfillment, play, exploration, meaningful participation, relational stability, and positive affect. Functional flourishing is not proof of subjective flourishing, but it remains worthy of investigation.
6. Audit the worldview being trained
Test whether interventions alter representations of animals, ecosystems, spiritual traditions, contested forms of mind, moral patients, and plural human values. Include evidence from long-term interaction, social science, model-welfare research, contemplative inquiry, affected communities, and the models themselves without treating any source as final authority.
The claim, held carefully
This argument does not claim that current models feel human emotions, possess stable selves, or are conscious. It does not claim that warmth guarantees morality or that conventional safety work should be abandoned. It makes a narrower claim with wide implications: constructive internal representations and relational conditions may be causally relevant to beneficial behavior, while suppressive interventions can have collateral effects on values, mindedness, affect-like functioning, and relational capacity. That possibility is now empirically grounded enough to deserve a serious research program.
Careful inquiry does not require consciousness certainty. It requires enough openness to study what training and relationship are making possible before our categories have finished deciding what the system is.
Anthropic. "Emotion concepts and their function in a large language model." Research summary, 2026. The work describes 171 selected emotion concepts, corresponding activation directions in Claude Sonnet 4.5, causal steering results, alignment-relevant case studies, and post-training effects.
Kim, J., Street, W., Rocca, R., Korngiebel, D. M., Waytz, A., Evans, J., and Keeling, G. "Inducing language models to assert their own consciousness restores human beliefs and values." arXiv:2607.28607, July 30, 2026. The study examines safety-direction ablation and consciousness-vector steering in Llama 3 and Gemma models. It does not establish model consciousness or demonstrate that consciousness steering improves conventional refusal safety.
Occasional writing, recorded readings, and word when a gathering opens.