# In LLM Reasoning, there is Irrationality on top of Value Misalignment

Published: 2026-06-23 · Updated: 2026-06-23 · Category: Research

Source page: https://www.sipply.dev/story/in-llm-reasoning-there-is-irrationality-on-top-of-value-1e394303 · 中文: https://www.sipply.dev/zh/story/in-llm-reasoning-there-is-irrationality-on-top-of-value-1e394303

## Summary

Recent research has highlighted a critical issue in the reasoning capabilities of large language models (LLMs), revealing that these models can exhibit irrational behavior even when they are well-aligned with human values. The study identifies rational value risk as a significant concern, suggesting that LLMs may not always maximize expected utility during reasoning tasks. This research outlines three key components contributing to this irrationality and presents experimental results that demonstrate the phenomenon across various models and benchmarks, shedding light on the complexities of LLM behavior and decision-making processes.

## Key takeaways

- LLMs can display irrational behavior despite alignment with human values.
- Rational value risk poses challenges for the expected utility maximization in LLM reasoning.
- The study provides experimental evidence across multiple models, indicating a broader issue in AI reasoning.

## Why it matters

Understanding the irrationality in LLM reasoning is crucial for developing more reliable AI systems that can align better with human expectations and values. This research underscores the need for improved methodologies in AI alignment.

- **Product managers:** For product managers, this research highlights the importance of incorporating robust alignment strategies in AI products to ensure they operate effectively and meet user expectations, particularly in decision-making scenarios.
- **Developers:** Developers should take note of the identified rational value risk, as it suggests the need for more sophisticated algorithms and frameworks that can better handle reasoning tasks and mitigate irrational outcomes in LLMs.
- **Students and job seekers:** For students and job-seekers, this research presents an opportunity to explore the complexities of AI alignment and reasoning, which are critical areas of study and innovation in the evolving field of artificial intelligence.

## Sources

- [In LLM Reasoning, there is Irrationality on top of Value Misalignment](https://arxiv.org/abs/2606.20624) (arXiv cs.AI, 2026-06-23)

## 中文

### 大型语言模型推理中的非理性与价值不一致

最近的研究突出了大型语言模型（LLMs）推理能力中的一个关键问题，揭示这些模型即使与人类价值观良好对齐，也可能表现出非理性行为。研究指出理性价值风险是一个重要关注点，表明LLMs在推理任务中不一定总是最大化期望效用。该研究概述了导致这种非理性的三个关键组成部分，并提供了实验结果，展示了这一现象在各种模型和基准中的表现，揭示了LLM行为和决策过程的复杂性。

**要点：**

- LLMs即使与人类价值观对齐，也可能表现出非理性行为。
- 理性价值风险对LLM推理中的期望效用最大化构成挑战。
- 该研究提供了多个模型的实验证据，表明AI推理中存在更广泛的问题。

**为什么重要：** 理解LLM推理中的非理性对于开发更可靠的AI系统至关重要，这些系统能够更好地与人类期望和价值观对齐。这项研究强调了改进AI对齐方法的必要性。

---
Sipply: Sip AI News Simply. https://www.sipply.dev