100%

Embedding quality + clone fingerprints

Goal

Make embedding-based semantic-dup real by extending the existing §06 embedding stack: enrich embedding_text, eval-gate a code-specific model against the current UniXcoder baseline, and add AST clone fingerprints fused with embeddings for type-4 detection.

Builds on the existing §06 stack (not greenfield)

A full §06 embedding-quality stack already ships in intel_repo and this section EXTENDS it; it does not rebuild it.

  • intel_repo/neo4j/enrich_embeddings.py — build_symbol_text + the UniXcoder embedding pipeline. DEFAULT_MODEL = "microsoft/unixcoder-base" (enrich_embeddings.py:61); the 768-dim UniXcoder vectors back the symbol_embedding HNSW index. UniXcoder IS the current model.
  • intel_repo/tests/embedding_golden_seeds.json (30-pair) + intel_repo/tests/test_embedding_quality.py — the labeled SIMILAR-RETRIEVAL sample + harness gating Hit-rate@5 ≥ 0.6 (load-bearing) + MRR@10 ≥ 0.4 (intel_repo/CLAUDE.md:345-356).
  • The golden-seed sample is a similar-retrieval sample, NOT a clone/dup sample — w-642c4d0d fills the genuine clone/dup ground-truth gap, mirroring the golden-seed shape.
  • The §06.0 model decision tree (intel_repo/CLAUDE.md:380-390) already evaluated all-MiniLM-L6-v2, nomic-embed-text-v1.5, and microsoft/unixcoder-base; UniXcoder won (pre-gate MRR 0.702). The challenger this section eval-gates is jina-embeddings-v2-base-code against that recorded UniXcoder baseline.
  • intel_repo/neo4j/metrics.py Score B carries the type-4 banded cosine fusion (cosine_floor = 0.80 / cosine_high_band = 0.92); the type-4 work below adds the AST structural term, it does not re-invent the fusion.

Implementation Sketch

  • Extend the tree-sitter extractor to capture the signature + leading doc-comment, then enrich build_symbol_text with the NORMALIZED form (content/decisions/01-embedding-enrichment-staleness.md) so the §06.3 deterministic-staleness invariant holds.
  • Build a labeled clone/dup sample (type-1/2/3/4 pairs) over the Ori corpus; eval-gate jina-embeddings-v2-base-code against the current UniXcoder baseline and record precision/recall — the numbers pick the model.
  • Add normalized AST Merkle-hash fingerprints (type-1/2/3) as an extraction side-channel; fuse the structural fingerprints with embedding KNN for type-4 semantic clones.

Work Items

  • Enrich embedding_text (enrich_embeddings.py:build_symbol_text) with the NORMALIZED symbol signature + leading doc-comment; extend the tree-sitter extractor to capture them. The build_symbol_text docstring + §06.0 task (a) record that the extractor stores neither today; normalization (strip param names, keep arity + type shape; collapse doc-comment to the leading sentence) bounds the embedding_content_hash churn so the §06.3 deterministic-staleness invariant (intel_repo/CLAUDE.md:323,447) is preserved per content/decisions/01-embedding-enrichment-staleness.md.
  • Build a labeled clone/dup evaluation sample over the Ori corpus carrying type-1/2/3/4 pairs as fingerprint ground truth, mirroring intel_repo/tests/embedding_golden_seeds.json (the harness intel_repo/tests/test_embedding_quality.py reads it). This is the clone/dup sample the golden-seed retrieval sample is NOT.
  • EVAL-GATE the jina-embeddings-v2-base-code candidate against the current UniXcoder baseline (DEFAULT_MODEL, enrich_embeddings.py:61) over the w-642c4d0d labeled sample; record precision/recall; the numbers pick the model. The candidate set is jina-code (the genuine new candidate) vs the recorded §06.0 UniXcoder baseline — never UniXcoder against itself.
  • Add AST Merkle-hash clone fingerprints (type-1/2/3) as a zero-cost tree-sitter extraction side-channel (normalized subtree hash). Fingerprints come from the tree-sitter extractor here, NOT from the compiler — the SCIP-from-oric fingerprint route was superseded into plans/completed/scip-emission (which carries no fingerprint work), so this section is the SOLE owner of AST clone fingerprints (see mission.md Strategic anchor).
  • Fuse the AST structural fingerprints (w-4409fbab) with embedding KNN (the w-f355c352 eval-gated model) for type-4 semantic clones. This item CONSUMES both w-4409fbab and w-f355c352 outputs; it is served only after both land (see Work-item ordering).

Work-item ordering

  • w-9a38f575 (enrichment) and w-642c4d0d (labeled clone sample) are independent and may proceed in parallel.
  • w-f355c352 (eval-gate) depends on w-642c4d0d (it scores models against that sample).
  • w-4409fbab (AST fingerprints) depends on w-642c4d0d (it validates fingerprint recall against the labeled clone pairs).
  • w-3d19af41 (type-4 fusion) depends on BOTH w-4409fbab (structural term) AND w-f355c352 (embedding-KNN model) — it is the terminal item and must not be served before its inputs exist.

Design notes

  • Type-2 normalization preserves the data-driven extractor design by mapping language-specific leaf nodes (identifiers, literals) to canonical placeholders before hashing — the normalization table is data, not per-language code.
  • A single Merkle subtree hash catches type-1/2 clones but misses type-3 gapped clones; use a set-based MinHash over the normalized subtree-hash set so inserted/deleted statements still match above a Jaccard threshold.

Cross-section dependency

  • w-3d19af41 (type-4 fusion) builds on the HNSW-cosine substrate delivered by semantic-dup-correctness (s-a04ca9c6, complete) and on the §06 embedding pipeline. The substrate is complete, so this section is reachable and unblocked; the DAG records s-a04ca9c6 as the predecessor-output dependency.

References

  • intel_repo/neo4j/enrich_embeddings.py (build_symbol_text, DEFAULT_MODEL :61); intel_repo/tests/embedding_golden_seeds.json + intel_repo/tests/test_embedding_quality.py; intel_repo/neo4j/metrics.py Score B banded fusion.
  • intel_repo/CLAUDE.md §06.0 model decision tree (:380-390), §06.3 deterministic-staleness invariant (:323,447), Quality gates (:345).
  • jina-embeddings-v2-base-code; CoIR clone-F1 benchmarks.
  • content/decisions/01-embedding-enrichment-staleness.md (enrichment-vs-staleness ADR).