doctor_frankentui

doctor_frankentui is the operator-facing verification harness. It packages everything the testing stack needs to turn a change into a pass/fail decision: replay capture, environment certification, suite orchestration across profiles, determinism soak, coverage gating, and semantic-contract validation. It is the binary CI runs on every push and the binary you run locally when a screen starts misbehaving and you need evidence.

Source: crates/doctor_frankentui/ (version 0.3.1, publish = false).

Why it exists

The testing stack in ftui-harness provides primitives — ProgramSimulator, ShadowRun, snapshot macros. That is enough for a developer writing a test, but not enough for a system operator evaluating a migration. Operators need:

• Reproducibility — identical runs across machines and points in time.
• Evidence — structured JSON + JSONL manifests you can diff, archive, and grep.
• Gating — an exit code that encodes go / no-go for release.
• Triage — a compact failure summary pointing to the first broken step.

Every one of these is an integration concern, not a unit-test concern. doctor_frankentui owns that integration.

What it integrates

• doctor_frankentui responsibilities
  • capture — TUI replay/capture through a PTY, writes RunMeta
  • seeding — MCP bootstrap + deterministic demo seeding
  • suite — profile-matrix replay (modern/tmux/dumb/…)
  • certify — environment smoke test + degraded-mode classification
  • report — HTML/JSON/plaintext synthesis over completed runs
  • plan — import-intake planning and snapshot materialisation
• Plus these side channels

Pipeline

Each stage writes structured artifacts into a shared run root (defaults under /tmp/doctor_frankentui/…). The next stage consumes what the previous stage produced; triage runs last and extracts the first N failure signals.

Commands at a glance

Every subcommand accepts --machine {auto, human, json} for CI-friendly output.

Artifact contract (at a glance)

Every top-level flow writes into a well-known layout. The schemas are versioned and fixed — artifacts has the full reference, but the short version is:

/tmp/doctor_frankentui/e2e/happy_<TIMESTAMP>/
├── meta/
│   ├── summary.json
│   ├── artifact_manifest.json
│   ├── events.jsonl
│   ├── command_manifest.txt
│   ├── env_snapshot.txt
│   └── tool_versions.txt
└── logs/

/tmp/doctor_frankentui/e2e/failure_<TIMESTAMP>/
├── cases/<case_id>/…
└── meta/
    ├── summary.json
    ├── case_results.json
    ├── replay_triage_report.json
    └── events.jsonl

/tmp/doctor_frankentui/determinism_soak_<TIMESTAMP>/
└── meta/
    ├── run_index.tsv
    ├── determinism_report.json
    └── determinism_report.txt

target/doctor_frankentui_coverage/
├── coverage_summary.json
├── coverage_gate_report.json
└── coverage_gate_report.txt

Prerequisites

• cargo — Rust 1.74+ toolchain (edition 2024).
• python3 — 3.11+ for TOML parsing via tomllib; otherwise 3.9+ with tomli installed (pip install tomli).
• jq — used by determinism-soak and a few report helpers.
• rg (ripgrep) — used by doctor_frankentui_no_fake_gate.py.
• cargo-llvm-cov — required for coverage gates (cargo install cargo-llvm-cov).

Typical local invocation

cargo run -p doctor_frankentui -- certify --full

./scripts/doctor_frankentui_happy_e2e.sh

./scripts/doctor_frankentui_failure_e2e.sh
python3 ./scripts/doctor_frankentui_replay_triage.py \
    --run-root /tmp/doctor_frankentui/e2e/failure_<ts> \
    --max-signals 8

./scripts/doctor_frankentui_coverage.sh

Related