Semantic Search — Engineering Design
Opt-in natural-language and keyword search over indexed function symbols: binary-quantized code embeddings (default code-daemon), Hamming retrieval, optional late fusion with blast radius, PageRank, name overlap, and eager token-bloom sketches from discover.
Figure 1: Dashboard Search tab — Ready badge, query controls (late fusion, keyword AND), and ranked hit table.
1. Goals
| Goal | How |
|---|---|
| Find functions by intent, not exact names | code-daemon ONNX embeddings (256-d MRL default) |
| Keep discover lean | Separate .rgbuilder/semantic_index.bin — built via semantic index |
| Fast retrieval at scale | Sign-quantized vectors + Hamming top-k |
| Blend structure + semantics | Late fusion re-ranks Hamming pool with graph signals |
| Agent-ready output | -f json semantic query + HTTP /api/semantic/query |
| Incremental rebuilds | Reuse rows when code_hash unchanged |
2. Architecture overview
flowchart TB
subgraph discover["discover"]
G[graph.snapshot.bin]
AR[analysis_results.bin]
SK[token bloom sketches per function]
G --> AR
G --> SK
end
subgraph semantic_index["semantic index (opt-in)"]
EX[semantic_extract body tokens]
EM[code-daemon / vocab / hash / ONNX]
DIF[optional call-graph Jacobi diffuse]
BIN[semantic_index.bin Hamming rows]
EX --> EM --> DIF --> BIN
end
subgraph query["semantic query"]
H[Hamming candidate pool]
F[late fusion re-rank]
HY[optional hybrid expand]
H --> F --> HY
end
discover --> semantic_index
BIN --> query
AR --> F
SK --> F
Two-stage retrieval (semantic_fusion.rs): Hamming pre-filter (default pool 256) → weighted fusion of semantic, blast, centrality, name, and sketch scores.
3. Embedders
| Embedder | CLI | Notes |
|---|---|---|
| code-daemon (default) | --embedder code-daemon | Bundled ONNX + SentencePiece in rgbuilder-analysis/assets/; requires semantic-onnx feature (default) |
| vocab | --embedder vocab | Compiled bag-of-tokens (vocab-accumulate-v1); offline, no ONNX; native 256-d |
| sign-hash | --embedder hash | Deterministic FNV sign-hash — CI / --no-default-features |
| custom ONNX | --embedder onnx --model PATH | Optional --tokenizer for SentencePiece |
Default dimensions: 256 (MRL for code-daemon; vocab native width). Clone needs git lfs pull for ~206 MB code-daemon weights.
Index-time diffusion (opt-in): --diffuse runs Jacobi mixing over the call graph on dense f32 buffers (CallGraph-sized, one scratch) before sign quantization. Defaults: α=0.25, 2 iterations, callees-only (--diffuse-bidirectional for callers+callees). Query does not re-diffuse — structure is baked into bits. --diffuse always re-embeds (skips the pure incremental bit-reuse shortcut) so bits reflect the diffused dense vectors. Extra RSS ≈ n_functions × dims × 4 × 2 bytes (≈ 3.8 GB peak buffers at 1.86M × 256); keep --diffuse off on huge repos until profiled.
Escape hatch for builds without ONNX:
cargo build --release --no-default-features
rg-build semantic index --embedder vocab # preferred offline
# or: rg-build semantic index --embedder hash
4. Structural sketches (Phase A)
At discover/extract time, each function gets a 256-bit token bloom (structural_sketch.rs) over declaration + body tokens. Sketches are stored on graph nodes and used for:
- Keyword AND filter — every query token must hit metadata or sketch
- Fusion term — Jaccard-style overlap between query tokens and sketch
No extra index pass required beyond normal discover.
5. Rust implementation map
| Component | Path |
|---|---|
| Index + Hamming search | crates/rgbuilder-analysis/src/semantic_search.rs |
| Vocab accumulator | crates/rgbuilder-analysis/src/semantic_vocab.rs |
| Call-graph diffusion | crates/rgbuilder-analysis/src/semantic_diffuse.rs |
| Body token extraction | crates/rgbuilder-analysis/src/semantic_extract.rs |
| Late fusion | crates/rgbuilder-analysis/src/semantic_fusion.rs |
| Hybrid expansion | crates/rgbuilder-analysis/src/semantic_hybrid.rs |
| Bundled code-daemon | crates/rgbuilder-analysis/src/semantic_embedded.rs |
| ONNX runtime path | crates/rgbuilder-analysis/src/semantic_onnx.rs |
| Token bloom at extract | crates/rgbuilder-graph/src/structural_sketch.rs |
| CLI | src/cli/semantic.rs, semantic_output.rs |
| HTTP API | src/cli/semantic_api.rs, http_serve.rs |
| Manifest export | crates/rgbuilder-export/src/manifest.rs |
6. Dashboard implementation
| Piece | Path |
|---|---|
| Tab | dashboard/src/SearchView.tsx |
| HTTP client | dashboard/src/semanticSearch.ts |
| Status + query API | GET /api/semantic/status, POST /api/semantic/query |
Requires rg-build serve (not static python -m http.server) so the semantic API is available.
7. CLI usage
rg-build discover .
rg-build semantic index # default code-daemon, 256-d
rg-build semantic index --incremental # reuse unchanged code_hash rows
rg-build -f json semantic query "shopping cart checkout" --limit 10
rg-build -f json semantic query "OrderService" --keyword-and
rg-build -f json semantic query "auth login" --expand neighbors --expand-depth 2
# Fusion is on by default; use --no-fusion for pure Hamming
rg-build serve --open # dashboard Search tab + /api/semantic/*
Index-only flags: --embedder, --dimensions, --model, --tokenizer, --incremental,
--diffuse / --no-diffuse, --diffuse-alpha, --diffuse-iters, --diffuse-bidirectional.
Query flags: --no-fusion (fusion is on by default), --keyword-and, --candidate-pool, --expand, --expand-depth.
# Offline / no ONNX (preferred)
rg-build semantic index --embedder vocab
rg-build semantic index --embedder vocab --diffuse
8. On-disk artifacts
| Path | Content |
|---|---|
.rgbuilder/semantic_index.bin | Quantized embeddings + metadata (schema v2) |
.rgbuilder/dashboard/manifest.json | semantic section when index present |
9. Testing
| Layer | Location |
|---|---|
| Hamming + index roundtrip | crates/rgbuilder-analysis/src/semantic_search.rs tests |
| Vocab accumulate | crates/rgbuilder-analysis/src/semantic_vocab.rs tests |
| Call-graph diffusion | crates/rgbuilder-analysis/src/semantic_diffuse.rs tests |
| Fusion scoring | crates/rgbuilder-analysis/src/semantic_fusion.rs tests |
| QE oracles | tests/semantic_search_qe.rs |
| Multi-query timing | tests/semantic_query_timing.rs (polyglot CI; linux ignored + RGBUILDER_LINUX_SEMANTIC=1, prefer --release) |
| CLI subprocess | tests/cli_output/subprocess_golden_path.rs |
| HTTP semantic API | src/cli/http_serve.rs unit tests |
Time linux-scale Hamming with a release build — debug can be ~100× slower on the scan. Index load is dominated by bincode deserialization of per-function string metadata (~tens of seconds at ~1.8M rows); query after load is a few milliseconds in release.
Regenerate screenshots:
rg-build -r ~/git/java/gbuilder semantic index
rg-build -r ~/git/java/gbuilder serve --port 8080
DASHBOARD_URL=http://127.0.0.1:8080/ node dashboard/scripts/capture-design-screenshots.mjs
Demo video (5 s per feature, tab + panel highlighted):
DASHBOARD_URL=http://127.0.0.1:8080/ node dashboard/scripts/record-feature-demo.mjs
10. Related docs
- Blast radius design — fusion blast term + hybrid
--expand blast - Graph metrics design — PageRank centrality term
- HTTP API —
/api/semantic/* - CLI / JSON API — semantic JSON shapes