Verify Existing Claims

Verify claims in an academic paper

Preserve the original sources, choose checks appropriate to each claim, and report what the recorded evidence supports.

From a paper to bounded findings

Record the exact paper and directly referenced sources, byte identities, retrieval attempts and substantive review coverage.
Extract atomic claims with their source locations, conditions, population, units and scope.
Select an appropriate check and freeze a versioned protocol before each main campaign.
Execute a registered implementation or import documented external evidence. Preserve successful, failed and excluded runs.
Review the evidence under tested conditions. Preserve original assessments when later findings change a label.
Give every unresolved claim a next check, dependency, feasibility estimate, decision rule and stopping rule.

Methods appropriate to the claim

Numerical: Match exact inputs, configurations, units, denominators and observation identities; compare with a declared tolerance.
Algorithm: Use independent references, invariants and boundary cases; finite agreement is not a proof of universal correctness.
Association: Audit cohorts, weights and dependence; do not upgrade descriptive associations into causal effects.
Causal: Use an intervention or justified causal design with explicit identifying assumptions.
Generalization: Define the target population and evaluate relevant new environments, locations or epochs.
Operational: Measure end-to-end operation, failure behavior, correctness and resource use.
Theory / standards: Examine the proof or primary normative text; mark empirical checks inapplicable where appropriate.

The initial built-in adapter performs deterministic matched numerical comparisons. It cannot conduct arbitrary experiments or establish causality, compliance or operational effectiveness. Domain implementations and operator imports remain explicit.

Protocol and evidence preservation

Declare questions, hypotheses, metrics and units; exact inputs and configurations; dates, algorithms and seeds or their inapplicability; controls, sensitivity, replication units and sample-size rationale; exclusions, missing-data treatment, dependence, statistics, resource limits, prior exposure and stopping rules. Separate pilots, main campaigns and follow-ups.

Source retrieval is distinct from complete review. A hash confirms identity; it does not establish substantive examination. Shared inputs can share upstream errors. Descriptive ranges are not confidence intervals; unsupported inference remains unsupported.

reproduced: The declared check meets its criterion under recorded conditions; broader validity is not implied.
discrepant: Examined evidence conflicts with the claim or declared criterion; comparability limits remain explicit.
inconclusive: Relevant evidence was examined but cannot resolve the claim within the declared scope.
untested: No adequate check has been completed; a plan or citation is not a completed assessment.

Dependency-aware claim queues

Claims track conclusions and campaigns execute checks. Queue existing-job prerequisites, choosing complete recorded runs or reviewed claim evidence. Registered numerical and networking campaigns run with at most four global jobs and two per assessment. Declared exclusive resources cannot overlap; undeclared interference remains a coverage limit.

Workers preserve original inputs, frozen protocols, results and attempt history. Failed or expired attempts require explicit follow-ups. Source review, proofs, standards and experiments requiring unavailable equipment remain manual. Networking experiments use frozen executable specifications and isolated experiment machines. Completing a campaign never automatically upgrades a scientific assessment.

Explore an illustrative claim queue

Synthetic teaching example. These are illustrative claims, statuses and timestamps; no research paper was evaluated.

Claims track scientific conclusions; campaigns execute checks. One job may supply evidence for several claims. Completion never automatically changes an assessment.

1 active jobs in this assessment / 2 assessment slots · global limit 4 · 5 jobs · 5 preserved events. No automatic experiment retries. Parallel execution does not establish independent evidence or review.

Scheduled workers are disabled in this environment. Jobs can be planned and preserved; automatic execution awaits worker activation.

Claim progress (6 claims; showing up to 50 per page)
Claim	Scientific assessment	Queued work
Example: compare reported latency cells Illustration; no paper source · numerical	Untested	Shared numerical campaign: running
Example: compare another latency cell in the same campaign Illustration; no paper source · numerical	Untested	Shared numerical campaign: running
Example: validate a causal explanation Illustration; no paper source · causal	Untested	Causal intervention requiring equipment: blocked
Example: interpret a normative standard Illustration; no paper source · standard	Untested	Substantive standards review: blocked
Example: review recorded discrepant evidence Illustration; no paper source · numerical	Untested	Numerical evidence awaiting review: awaiting review
Example: investigate an expired attempt Illustration; no paper source · numerical	Untested	Expired worker attempt: expired

Campaign jobs

Shared numerical campaign

running · priority 1

Illustrative numerical execution

Job 00000000-0000-4000-8000-000000000070 · automatic · last recorded change 2026-10-07T00:00:00Z

Exclusive resources: machine:example-host

Causal intervention requiring equipment

blocked · priority 2

Equipment and an intervention design are required

Automatic execution is inapplicable; a domain operator is required.

Job 00000000-0000-4000-8000-000000000071 · manual · last recorded change 2026-10-07T00:00:00Z

Requires reviewed evidence from 00000000-0000-4000-8000-000000000070

Substantive standards review

blocked · priority 3

Primary normative text needs substantive examination

Automatic execution is inapplicable; a domain operator is required.

Job 00000000-0000-4000-8000-000000000072 · manual · last recorded change 2026-10-07T00:00:00Z

Numerical evidence awaiting review

awaiting review · priority 4

Recorded discrepancy needs claim-specific review

Job 00000000-0000-4000-8000-000000000073 · automatic · last recorded change 2026-10-07T00:00:00Z

Recorded run 00000000-0000-4000-8000-000000000090: complete. Scientific discrepancies are findings; review remains separate.

Expired worker attempt

expired · priority 5

Lease expired; outcome uncertain, no automatic retry

Job 00000000-0000-4000-8000-000000000074 · automatic · last recorded change 2026-10-07T00:00:00Z

Recorded event history (5; latest 50 shown)

… · illustration · expired
Job 00000000-0000-4000-8000-000000000074: Lease expired; outcome uncertain, no automatic retry
```
{
  "data_origin": "software-fixture",
  "measured_charges_usd": null
}
```
… · illustration · awaiting review
Job 00000000-0000-4000-8000-000000000073: Recorded discrepancy needs claim-specific review
```
{
  "data_origin": "software-fixture",
  "measured_charges_usd": null
}
```
… · illustration · manual
Job 00000000-0000-4000-8000-000000000072: Primary normative text needs substantive examination
```
{
  "data_origin": "software-fixture",
  "measured_charges_usd": null
}
```
… · illustration · manual
Job 00000000-0000-4000-8000-000000000071: Equipment and an intervention design are required
```
{
  "data_origin": "software-fixture",
  "measured_charges_usd": null
}
```
… · illustration · running
Job 00000000-0000-4000-8000-000000000070: Illustrative numerical execution
```
{
  "data_origin": "software-fixture",
  "measured_charges_usd": null
}
```

All rows, identities and timestamps in this example are synthetic illustrations. No experiment was executed and no independent review is asserted.

Publication and privacy

Publication requires a scientific scope review and passed database, backend, frontend, provenance and access-control checks. Incomplete scientific coverage is publishable when its limits and actionable gaps are visible. Scientific discrepancies are findings, not software verification failures.

Published snapshots are public read-only. Later drafts, account identities and original private artifacts stay private. Evidence, reviews and publication history are append-only. AI and human reviews are labelled explicitly; development cases do not become held-out validation through reuse.

Existing evaluated assessment chats remain available in their own modules. A generic model prompt is not enabled without its own declared answer evaluation and publication criteria.

Methodological guidance: NIST experimental design · Center for Open Science preregistration.