CLI I/O sanity audit

Engineers use this document to understand what the CLI I/O test suites verify, how the subprocess harness works, and where to add coverage when changing serializers or flags.

The goal is a stable contract for:

• Human operators — text on stdout/stderr, progress on stderr, sensible exit codes.
• Machine consumers — deterministic JSON with versioned schemas, omitted keys instead of null, and composable topology arrays.

Test architecture (three layers)

flowchart TB
  subgraph L1["Layer 1 — Unit schema tests"]
    U["tests/cli_output/*.rs"]
    S["src/cli/*_output.rs fixtures"]
    U --> S
  end

  subgraph L2["Layer 2 — Golden-path subprocess"]
    G["subprocess_golden_path.rs"]
    G --> B["CARGO_BIN_EXE_rg_build"]
  end

  subgraph L3["Layer 3 — Full-platform subprocess"]
    A["all_commands_sanity.rs"]
    A --> B
  end

  F["tests/fixtures/tiny_polyglot_repo"]
  G --> F
  A --> F

  L1 -.->|"fast, no binary"| L2
  L2 -.->|"narrow regressions"| L3

Run everything:

cargo test --test cli_output --test subprocess_golden_path --test all_commands_sanity

CI: Maintainers order .github/workflows/ci.yml via workflow_dispatch or by adding the ci label on a PR. That job runs format/clippy, workspace tests, named QE steps (map_collision_qe, graph_correctness, semantic_search_qe, cross_feature_qe), and the three CLI I/O targets plus blast-radius perf. There is no automatic run on every PR open.

Individual targets:

cargo test --test cli_output              # serializers only
cargo test --test subprocess_golden_path  # discover + blast-radius golden paths
cargo test --test all_commands_sanity     # comprehensive subprocess audit

Subprocess harness (all_commands_sanity.rs)

Design goals

1. Never touch a developer tree — each test copies tests/fixtures/tiny_polyglot_repo into a tempfile::TempDir.
2. Explicit sandbox database — graph writes go to {temp}/sandbox_graph.db via -d, not {repo}/.rgbuilder/.
3. Real binary — uses env!("CARGO_BIN_EXE_rg_build") so cargo test always runs the binary built for the current profile.
4. Shared repo root — -r {temp_repo} keeps paths stable for slice/inspect file arguments.

Sandbox helper

Assertion helpers

Single test: test_all_cli_commands_json_schema_sanity

Execution order inside the test (each phase uses a fresh discover ingest unless noted):

Separate test: test_discover_cli_flags — --exclude, -v, --security, --with-cfg, --with-taint.

CLI note: inspect takes a layer subcommand (inspect SYMBOL dom), not --layer dom.

Fixture: tests/fixtures/tiny_polyglot_repo

Minimal polyglot repo used by all subprocess suites.

Known limits (documented so engineers do not chase false failures):

1. Rust Calls edges — Rust plugin may not emit call edges in this tiny fixture; blast-radius/check upstream counts for Rust symbols can be zero.
2. Duplicate bare names — process, helper, etc. exist in both languages; blast-radius needs Class::method or --class; check skips ambiguous symbols via resolve_unique_symbol. Use publishEvent for subprocess scale-failure coverage.
3. Re-discover after cache schema changes — subprocess tests always run fresh discover; stale local .rgbuilder/ is not used.

Layer 1 — Unit schema tests (cargo test --test cli_output)

These tests call serializer fixtures in src/cli/*_output.rs directly. They do not spawn the CLI. Add or extend a test here when changing JSON field names, optional-key rules, or fixture builders.

Module map

What each command’s unit tests prove

discover

• schema_version: 2, command: discover
• Metrics object always includes: files_discovered, files_indexed, files_skipped, nodes_generated, edges_generated, duration_ms
• build_discover_response maps PipelineStats → JSON fields correctly

blast-radius (v2)

• Top-level: target, metrics, topology, gatekeeping
• gatekeeping.handoffs is always a present empty array when slices skipped
• Topology caller entries expose id, fqn, file_path
• Target v2: language, canonical_fqn; signature omitted when None
• metrics.caller_depth_limit present only when --depth N passed; impact_zone_size matches filtered zone
• --depth N post-filters cached/engine impact zones by incoming call hops (see json-api.md blast-radius catalog)
• Unresolved UUIDs in cache → caller dropped from topology (nil-UUID guardrail)

gql

• schema_version: 1, rows, count, explain
• Row cells: binding, node, type, file
• Empty result → rows: [], not omitted

metrics

• Full response includes all three sections when built with data
• Pagerank-only build: betweenness and communities keys absent (not null)
• wrap_metrics_payload injects schema_version

check

• Root: policy, violations, passed
• Passing run: violations: []
• test_check_passed_false_contract: serializer contract for passed: false
• Subprocess: publishEvent + max_impact_nodes: 0 → exit 1 (see Layer 2 / Layer 3)

slice

• CFG view: view, nodes, edges arrays — not legacy scalar block counts
• blocks key must not appear in CFG JSON

inspect

• CFG layer fixture: symbol, layer, nodes, edges
• Nodes use stable block_index + start_line (not internal debug pointers)

Layer 2 — Golden-path subprocess (subprocess_golden_path.rs)

Focused regressions that proved fragile during P2 work. Uses the same temp-copy fixture pattern but default -d (graph under {repo}/.rgbuilder/) except where noted.

Add a golden-path test when a specific discover → command pipeline breaks in production but unit fixtures still pass.

Global platform rules (enforced where marked)

Architecture alignment: Code_structure.md — CLI thin, serializers in *_output.rs, cache enrichment in rgbuilder-analysis.

Coverage gaps

All items from the original audit matrix are now covered by subprocess and/or unit tests. When adding new CLI flags or JSON fields, extend:

• tests/cli_output/all_commands_sanity.rs — full-platform subprocess loop + test_discover_cli_flags
• tests/cli_output/subprocess_golden_path.rs — focused regressions
• tests/cli_output/*.rs — serializer unit fixtures

Future optional expansions (not required for baseline compliance):

Extending coverage

Changed a JSON field in *_output.rs

1. Update the typed struct and fixture_* builder in the same file.
2. Fix the matching module under tests/cli_output/.
3. If the field is user-visible in subprocess output, add an assertion to all_commands_sanity.rs or subprocess_golden_path.rs.

Added a new CLI JSON command

1. Create src/cli/<cmd>_output.rs with SCHEMA_VERSION constant and fixture.
2. Add tests/cli_output/<cmd>.rs and mod <cmd>; in main.rs.
3. Append a step to test_all_cli_commands_json_schema_sanity.
4. Document the schema in json-api.md field catalogs.

Added a subprocess-only flag

Prefer asserting in all_commands_sanity.rs if the flag affects JSON shape or exit code; use subprocess_golden_path.rs for one critical pipeline only.

Related docs

• json-api.md — field-by-field JSON reference (blast-radius + catalogs)
• json-api.md — programmatic parsing guide
• graph-storage-architecture.md — snapshot layout, blast lookup cache
• Code_structure.md — where to put CLI vs analysis changes