63%

bug-dedup-module

Goal

Detect when a freshly-filed bug duplicates an existing open bug, and refuse the duplicate or route it to enhance the existing entry.

Implementation Sketch

Deliver the bug-dedup-module slice of the graph-native relatedness + dedup capability per the plan mission; ground every claim against the verified intel-graph facts in the overview.

Work Items

  • Add scripts/add_bug_runtime/bug_dedup.py: compute_bug_dedup(entry, open_bugs, *, closed_bugs=(), graph_query=…) -> BugDedupResult with verdict new | exact_duplicate | related_enhance, matched_bug_id, score, shared_symbols, closed_match_ids, rationale. Corpus SSOT = the authoritative bug-tracker JSON (partition open+in-progress candidates from closed_bugs, since related-nodes itself does not filter by status); the graph query RANKS via a shared, generalized query-composition + no-fire/degrade kernel — extract/generalize relatedness.py’s _compose_query_text and _query_related_plans (currently hardcoded —kind plan) into a —kind-parametrized helper reused for the new —kind bug query, instead of cloning a second composition path in bug_dedup.py.
  • Gate verdicts using the module’s OWN evidence-derived bands — never relatedness.py’s GRAPH_FIRED_THRESHOLD/GRAPH_COVERAGE_THRESHOLD, which are calibrated for the binary plan-fired question, not this three-way exact/related/new bug-dedup question (live probes: a verbatim self-refile scores 44.1-44.3, inside the sibling’s related band and below its self band). exact_duplicate is confirmed corpus-side (normalized title/repro text comparison against the top-ranked graph candidate; the top-to-second score RATIO, observed at 2.7x-4x on live self-matches, is a ranking signal only, never an absolute-score gate). related_enhance uses a dedup-owned live-distribution calibration test to set its own score band. same-subsystem is required but not sufficient to escalate (tooling-cli alone is half the live corpus) — corpus-side text confirmation carries the real discrimination. shared_symbols stays a bonus tier, never a verdict gate (empty in every live probe despite 119 resolvable chains). closed-bug matches -> informational regression hint only (closed_match_ids), never block.
  • Add a bug-dedup-owned live-distribution calibration evidence test (in the style of scripts/add_bug_runtime/tests/test_graph_parity_probe.py’s fixture-family + distribution-print pattern) over a self-refile / genuinely-related / unrelated fixture family (anchored on BUG-07-190’s recorded false-positive case and at least one verbatim self-refile), empirically deriving compute_bug_dedup’s own exact_duplicate and related_enhance score bands instead of importing or retuning relatedness.py’s GRAPH_FIRED_THRESHOLD/GRAPH_COVERAGE_THRESHOLD (which are constrained — calibrated for a different, binary question). This is the concrete evidence artifact the mission’s SSOT/no-heuristic-fudging posture requires before the gate logic in w-2f395ac9 can be trusted.
  • Wire compute_bug_dedup into append_open_bug so it runs independently of the existing skip_relatedness early-return (bug_tracker_io.py:146-148) — restructure that path so --skip-relatedness alone does NOT also skip dedup; dedup runs BEFORE the plan-relatedness check on every path. Add a dedup field to AppendResult (preserving the existing attribute/mapping getitem shim). Add exit_reasons duplicate_bug (no write) and enhanced_existing (enhance write), landing both in the CLI/runtime AND in .claude/rules/orchestration-rules.json’s add-bug exit-reason registry (both the detailed list at :2631-2635 and the parity list at :2706) in the same change, per the Decision-31 parity note in bug_tracker_io.py’s docstring. Add a --skip-dedup escape hatch independent of --skip-relatedness.
  • Add a dedup-check --entry-json read-only CLI subcommand to main.py emitting the verdict JSON; extend —self-test.

Fresh intel (regenerated)

The single most important fact: the code-symbol graph is structurally BLIND to this section’s terrain (the ori repo indexes Rust + Ori only — 0 Python symbols under scripts/**, verified), but the graph IS the section’s runtime dependency — related-nodes --kind bug is live, was probed end-to-end this session, and the live score distribution FALSIFIES the naive threshold design the section could otherwise inherit from its completed sibling.

The decisive [JOIN] (probes §3.3 x calibration constants §3.4): a VERBATIM exact re-file of open bug BUG-04-227 (title + repro + --subsystem) self-matches at score 44.3, BELOW the sibling’s GRAPH_COVERAGE_THRESHOLD = 52.0; a second verbatim self-match probe (BUG-02-023 title) scored 44.1; meanwhile a GENUINELY RELATED bug (BUG-03-011, computed-key map literal) scored 16.5, below GRAPH_FIRED_THRESHOLD = 35.0. Raw Lucene scores vary with input length and are NOT transferable across question types (fired-vs-not for plan relatedness vs exact-vs-related-vs-new for bug dedup). The work item’s own words — “Corpus SSOT = the authoritative bug-tracker JSON; the graph query RANKS” — are therefore load-bearing, not decoration: exact_duplicate MUST be confirmed corpus-side (normalized title/text comparison against the matched entry in open-bugs.json), with the graph as candidate RANKER only; the exact-duplicate signature in live data is the top-to-second score RATIO (2.7x-4x observed), never an absolute score.

Decisions this package forces before any code is written:

  1. Do not transfer GRAPH_FIRED_THRESHOLD/GRAPH_COVERAGE_THRESHOLD (relatedness.py:112/121) to the dedup verdicts. They were calibrated on the misroute fixture family for the plan-fired decision (co-worded-unrelated <= 28.5, related >= 41.5, min self-match 52.6 per tests/test_graph_parity_probe.py); this session’s independent self-match probes landed at 44.1/44.3 — inside the sibling’s “related” band, below its “self” band. Calibrate dedup’s own bands OR (better) gate exact_duplicate corpus-side.
  2. related-nodes does NOT filter by status — resolved/obe rows rank in every probe (§3.3). The candidate partition (open+in-progress = live dedup candidates; resolved/obe = regression-hint-only closed_match_ids) is the MODULE’s job, from the corpus JSON, matching the closed_bugs=() parameter in w-254adf39.
  3. The skip_relatedness=True early-return in append_open_bug (bug_tracker_io.py:146-148) bypasses everything — the w-3291206f wiring (“dedup BEFORE the plan-relatedness check”, separate --skip-dedup hatch) must restructure that path so --skip-relatedness alone does NOT skip dedup.
  4. shared_symbols is nearly always empty in practice — only 119 Bug->CodeReference->RESOLVES_TO->Symbol chains exist corpus-wide (87 of 984 bugs); every live probe returned shared_symbols: [] even with resolvable backticked symbols in the input. Treat it as a bonus tier, never a verdict gate (the sibling already encodes this posture — mirror it).
  5. The BugDedupResult envelope is a cross-plan contract: plans/workflow-engine-rework/mission.md:55 names bug_dedup’s typed verdicts as expected upstream engine inputs (user-sequenced: this plan executes first). Envelope fields (verdict/matched_bug_id/score/shared_symbols/closed_match_ids/rationale) should be treated as frozen once landed.
  6. “dedup” is an overloaded name in this exact package: ACTION_DEDUP_INTO_SECTION + exit_reason dedup_into_section (relatedness.py:83, __main__.py:205) already mean “bug subsumed by a feature PLAN section”. The new bug-vs-bug verdicts (duplicate_bug, enhanced_existing) must stay lexically and structurally distinct (§3H NAMING risk).

DIG DEEPER

scripts/intel-query.sh plan-status graph-native-bug-relatedness
python -m scripts.plan_corpus.read --summary plans/graph-native-bug-relatedness
scripts/intel-query.sh related-nodes --text "<any open-bug title+repro>" --kind bug --subsystem <s> --limit 5

(full dossier: bug-dedup-module—s-6924d386.intel.md)