Benchmarks, tests, and what they establish

LunarTrace separates historical benchmark recovery, software correctness tests, runtime qualification, and scientific evaluation. None substitutes for all the others.

Historical benchmark recovery

The existing historical_benchmark entry point and lunartrace benchmark command recover the admitted Chang’e 6 benchmark basis. The retained quantities include observation/pair counts, explicit frontier definitions, and whether the reference pair was retained under a variant.

This is a historical reference-recovery projection. It is not an independent replay from the original missing rows. Frontier retention is not labelled accuracy, recall, or validated scientific optimality.

Deterministic software tests

The repository contains tests for scientific models, partitioning, screening, artifact identity, source-response handling, contracts, and UI failure semantics. R2 adds authored tests for draft/query semantics, bounded discovery, geometry parsing, exploratory dominance, export auditing, and browser workflows.

A synthetic fixture is appropriate for verifying a predicate or error path. It must remain labelled and isolated from production discovery. It does not establish the actual lunar archive population or the scientific value of a particular image pair.

Runtime and browser qualification

Build success, unit tests, browser automation, visual inspection, and physical-device interaction are separate proof surfaces. Tests authored by a connector-only implementation agent are not tests executed on the target workstation.

A screenshot can reveal clipping or unclear hierarchy. It does not prove source-query closure, image registration, or scientific truth. A successful source request can establish that a bounded query ran; it does not establish that the interface is usable or that all relevant real-world evidence exists in that archive.

No invented performance results

This manual does not claim measured Lab R2 latency, GPU performance, live ODE reliability, or cross-browser conformance before the corresponding execution receipts exist. Local qualification should record the exact source revision, environment, query, and observed result.

Useful future empirical measures include time to a first source-backed result, query closure rate under bounded scopes, preservation of exact identities across interactions, and time from an exploratory selection to a reproducible export. These are evaluation directions, not current benchmark claims.

Reproduction commands

From an authorized source checkout with dependencies installed:

uv run lunartrace benchmark --repository-root .

The command returns structured JSON. Interpret its fields within the historical basis described in Chang’e 6: a retained historical case, not as a new lunar-science result.