Don't Bet on One Chain

The principle

A single greedy chain of thought is fragile. Sample several independent reasoning paths and take the majority answer, and you get large, consistent gains. Correct reasoning tends to converge while mistakes scatter, so agreement across independently generated plans is a real signal worth trusting before you act on something that matters.

Why it happens

One sampled reasoning path is one route through a probabilistic space. If it makes a bad early move, everything after it inherits that move. Multiple independent attempts give you a different signal: correct answers tend to converge, while mistakes scatter. Repeated sampling only helps when you can choose among the samples, through majority vote for comparable answers or through an external verifier for plans and artifacts. Use it for consequential, hard-to-reverse decisions. Do not spend 5x compute on routine steps that are cheap to undo.

Watch for

• High-stakes outputs ride on a single greedy generation with no second opinion.
• Re-running the same prompt yields meaningfully different answers, revealing the first one was luck.
• Errors slip through because nothing checks whether independent attempts actually agree.

Apply it

1. For consequential decisions, generate the answer several independent times instead of trusting the first.
2. Take the majority answer when outputs are comparable, or use an external check to pick among them.
3. Treat disagreement across the samples as a signal to escalate rather than silently picking one.

Sources and further reading

• Self-Consistency Improves Chain of Thought Reasoning · Wang et al., 2022
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling · Brown et al., 2024