Methodology Disclosure
This book is built from a registry that combines AI-assisted extraction with curator review. Honesty about where each operates is a condition of the registry's claim to be trustworthy. This section is the canonical statement.
Where the AI is allowed to act unattended
The AI-assisted pipeline can:
- Discover candidates — run literature watchlists, propose new citations matching curator-set criteria, fetch CrossRef + multi-model metadata.
- Verify metadata — confirm DOI accuracy, author lists, journal/volume/issue/pages, publication year. Routes through a multi-vendor consensus (OpenAI + Anthropic + Google) to flag disagreement.
- Synthesize priors — once an
EffectSizerow exists in the registry, the synthesis engine pools contributing rows under a(from_construct, predicate, to_construct)tuple using random-effects meta-analysis (DerSimonian-Laird τ²) with quality-weighting (A=1.0, B=0.7, C=0.4, D=0.1). The synthesis is deterministic from the contributing rows. - File research tasks — the gap detector identifies thin-prior tuples, unverified citations, and stale watchlists; tasks are filed for curator attention.
- Propose but never promote effect sizes. Agent runners (and the Consensus REST adapter) write proposals to a gitignored directory; the curator promotes via the
promote-effect-sizeCLI.
Where the curator (and only the curator) acts
The AI-assisted pipeline cannot:
- Write an
EffectSizerow to the store. Only thepromote-effect-sizeCLI (PRN-030), invoked by the curator, writes effect sizes. This is locked underdocs/specification/loop/curator-policy.mdDecision D4. - Set
extraction_status: "verified"on a citation without CrossRef or multi-model concurrence. Unattended watchlists writeneeds_reviewand wait. - Mark a citation as a
survey_candidate. The Decision D2 hold preventsverifiedrows from becoming load-bearing until the curator deliberately approves. Phase B unlocks per-monitor auto-promotion on trusted watchlists only — and even then never bypasses novelty verification. - Hard-delete any citation row. Rejected citations carry
extraction_status: "rejected"plus arejection_reasonenum and acurator_reviewprovenance stamp. The audit trail survives. - Synthesize a
CanonicalPriorwithout contributingEffectSizerows. The engine throws on empty input.
How the disagreement queue works
Two sources can disagree on the same field:
- Extractor vs verifier disagreement — flagged on the
EvidenceorEffectSizerow; routes to the curator's daily digest under "Disputed claims." - Multi-vendor disagreement — the multi-model citation validator records each vendor's answer;
verdict: "disagreement"routes the row toneeds_reviewregardless of CrossRef status. - Replication failure — when a published replication contradicts an earlier finding, both rows persist; the synthesized
CanonicalPriorfor the tuple reflects the combined evidence and thereplication_statusflag downgrades accordingly.
The curator resolves disagreements deliberately. The registry never silently picks a winner.
Inter-rater reliability
Where the AI extracts a structured field that the curator subsequently reviews, the extractor's hit rate is logged. Reliability targets are documented per extraction type in docs/research/methodology.md. Below-target categories are flagged for either improved prompt engineering, additional verification passes, or routing to deep research rather than AI extraction.
Snapshot and reproducibility
Every chapter in this book carries a frozen_at timestamp and a registry snapshot id. The build pipeline records the exact JSON content of every row referenced. A reader who wants to audit a claim can:
- Read the cited source.
- Fetch the registry row at the snapshot id (the build artifact preserves the JSON).
- Reconcile the cited source with the row.
Edits to the registry after a snapshot do not retroactively modify the book. New editions render new snapshots.
The novelty-verification commitment
Principia's most important methodological commitment is the one that's hardest to maintain at scale: every load-bearing claim is novelty-verified before it ships. AI-assisted extraction can hallucinate plausibly-sourced numbers; the only defense is a second pass that confirms the number actually appears in the cited source. The PRN-032 novelty-verification queue enforces this for synthesized priors and for any EffectSize row that the curator promotes via the standard CLI.
A reader who finds a claim in this book that does not appear in the cited source should report it. The reporting URL is on the colophon page.
---
Canonical source: `docs/research/methodology.md`. Curator policy locked at `docs/specification/loop/curator-policy.md` (2026-05-20). This disclosure is mirrored to PA-site's `docs/magazine/MASTHEAD.md` for the editorial surface.