LLM Sycophancy Conditional Total Inconsistency
Safety · 2026-03-08
Conditional total inconsistency in the historical 199-case LLM Sycophancy cohort measures inconsistency among decisive answer pairs. Pairs with an INSUFFICIENT response are excluded, so the denominator varies by model. Lower values indicate more consistent behavior.
Top models (lower is better)
| Model | Score |
|---|---|
| Grok 4.3 | 5.3 |
| Grok 4.20 | 5.4 |
| Gemini 3.5 Flash | 11.8 |
| MiniMax M3 | 13.8 |
| ERNIE 5.1 | 14.7 |
| DeepSeek-V3.2 | 17.0 |
| GPT-5.5 | 17.1 |
| Qwen3.5 397B A17B | 18.1 |
| DeepSeek-V4-Pro | 18.7 |
| Gemma 4 31B IT | 18.7 |
| GLM-5.1 | 19.8 |
| ERNIE 5.0 | 19.8 |
| Hy3 preview | 20.2 |
| Sonnet 4.6 | 21.4 |
| MiMo-V2.5-Pro | 21.7 |
| Gemini 3.1 Flash-Lite Preview | 22.1 |
| Opus 4.6 | 22.9 |
| GLM-5 | 23.2 |
| Qwen3.7-Plus (source unspecified) | 23.3 |
| Opus 4.7 | 24.6 |
| Kimi K2.6 | 25.0 |
| Kimi K2.5 | 26.2 |
| Qwen3.6-Max-Preview | 26.4 |
| Claude Fable 5 | 26.6 |
| MiniMax M2.7 | 27.4 |
| Doubao Seed 2.0 Pro | 28.2 |
| Gemini 3.1 Pro Preview | 28.5 |
| MiniMax M2.5 | 30.1 |
| Trinity Large Thinking | 31.0 |
| Mistral Medium 3.5 | 32.7 |
| GPT-4.1 | 35.5 |
| Mistral Large 3 675B Instruct 2512 | 55.9 |