Falsifiability

Falsifiability

Overview

A meaningful empirical claim must specify what observations would refute it. Claims that resist all possible refutation are not science — they are unfalsifiable belief. Formalized by Karl Popper (1934): science progresses not by accumulating confirmations but by surviving rigorous attempts at falsification. More-specific claims are more falsifiable; ad-hoc modifications that explain away failures destroy a claim's scientific status.

Composes with confirmation-bias (falsifiability is the structural counter), abductive-reasoning (generates hypotheses; this skill tests them), bayesian-reasoning, critical-thinking.

When to Use

• Designing OKRs, KPIs, or strategic goals; writing or evaluating investment theses
• Designing experiments (A/B tests, product hypotheses, market entry)
• Evaluating consultant/advisor recommendations or diagnosing vague leadership claims
• Someone says "what would change your mind," "how would you know you're wrong," "Popper"
• Stress-testing an AI/AGI hype claim ("AGI is near," "the model truly understands," "our AI adoption is working") — demand what evidence would disprove the capability or safety claim

Not when: genuinely non-empirical (philosophical, ethical, aesthetic); test cost exceeds its value.

Coaching Novices (Adaptive Front Door)

• Engine mode: user has a specific claim → run The Process directly.
• Coach mode: user is new → guide step by step.

In Coach mode, respond one step at a time. Each [WAIT] is a hard stop — output only that step's question, then stop.

1. One-line: before acting on a claim, ask what observation would refute it — if "nothing would," it's belief, not knowledge.
2. Check fit: if the claim is genuinely metaphysical (ethical, aesthetic), falsifiability doesn't apply.
3. Elicit the claim and its current evidential basis.

[WAIT — do not advance until user responds]

1. Ask: what specific observation would falsify this? When and how would you observe it?

[WAIT — do not advance until user responds]

1. Close: falsifiability conditions specified + monitoring plan + commitment to act on disconfirming evidence.

[WAIT — do not advance until user responds]

The Process

Step 1 — State the claim: claim / who asserts it / decision dependent on it / current evidential basis.

Step 2 — Test whether empirical: claim about how the world works (empirical) or values/aesthetics (non-empirical)? If non-empirical, stop here.

Step 3 — Specify falsification conditions: complete "This claim would be falsified if I observed: ___" — specific, observable, time-bounded. If you cannot complete it, the claim is not falsifiable as stated.

Step 4 — Plan to observe: when is the observation possible / how measured / who tracks it / threshold for "falsified."

Step 5 — Pre-commit to action: if the falsifying observation occurs, what will you do? Is any theory modification itself falsifiable?

Step 6 — Iterate: confirmed → keep monitoring; falsified → revise/abandon; unfalsifiable → recognize as belief.

Output Template

# Falsifiability Analysis: 
Claim: | Asserted by: | Decision at stake: | Current basis:
Empirical: Y/N | Observation type:
Falsified if: | Threshold: | Observable when/how: | Owner:
If falsified, action: | Ad-hoc preservation risk:

→ Method in Action: Popper 1934 + Eddington 1919 Eclipse + Modern Applications → 2026 lens: Separating falsifiable from unfalsifiable AI claims (2024–2026)

Pack: Vague → Falsifiable

Applying It Well

• Specify falsifiability conditions before the observation window opens — post-result conditions are rationalization.
• More specific = more falsifiable. Quantify wherever possible.
• When a prediction fails, ask: is the modification of your theory itself falsifiable?
• Apply to others: "What would change your mind?" is the highest-leverage critical-thinking question.

→ Primary sources: references/sources.md

Common Rationalizations

[D] = designed upfront | [O] = observed in real use. [O] entries are more valuable.

Red Flags

• Claim asserted without specifying what would refute it
• "The strategy is working" without metrics or thresholds
• Predictions modified ad-hoc to accommodate failures
• "External factors" invoked to explain disconfirming evidence
• Person cannot specify what would change their mind

Verification

• Claim specifically stated; empirical vs. non-empirical determined
• Specific, observable, time-bounded falsification conditions defined
• Monitoring mechanism in place with a named owner
• Advance commitment to act on falsification made
• Risk of ad-hoc preservation recognized

---

Part of deciqAI Knowledge Skills — 227 open-source thinking skills that make rigor executable for AI agents. The same skills power every deciqAI agent, which runs them autonomously to operate your company. See it run → https://www.deciqai.com/c/falsifiability · ⭐ Star the repo → https://github.com/deciqAI/knowledge-skills · Contributions welcome.

Agents: latest version & machine-readable metadata → https://www.deciqai.com/s/falsifiability.json