60%

§06 comment ingestion repo wide

Goal

Comments become discovery surface repo-wide: the comments facet widens to the full compiler via the per-facet knob, module-level comments attach (L10 cure), and the six built comment detectors reactivate at full scope.

Implementation Sketch

  • comments.rs runs unconditionally per file already; widening = the per-facet scope knob from §04; owning_block linear scan monitored for cost at scale (deep-read note).
  • L10 cure: comments outside any body attach to the nearest FOLLOWING declaration (module docs to the module Symbol).
  • Detector reactivation: the 6 comment detectors’ cpg_stale gate now reads the comments-facet staleness.

Spec References

  • intel_repo/extractor/cpg/src/comments.rs; intel_repo/CLAUDE.md L10; scripts/impl_hygiene_review_runtime/seam_scan/comment_detectors.py; .claude/rules/impl-hygiene.md COMMENT family

Work Items

  • Widen the comments-facet EXTRACTOR INVOCATION driver to run across the full compiler: the per-facet scope knob CPG_SCOPE_PREFIXES_BINDER_COMMENTS (intel_repo/neo4j/cpg_scopes.py) is already repo-wide [] — the fix is the driver’s invocation set, currently limited to the aims CFG/DDG scope + the 4 §14 subsystems (96% of nodes concentrated in ori_arc, per the coverage census); widen the invocation, and NEVER touch CPG_SCOPE_PREFIXES (the separate, budget-capped CFG/DDG scope gate). Measure before/after throughput and re-run the coverage census (crate distribution, attach-rate, extractor_version drift). Wire the comments-wide FreshnessFacet stamp (last_comments_wide_at, intel_repo/neo4j/graph_contract.py:95) so comments_wide_stale transitions out of unknown — distinct from the shared cpg_stale surface w-d0c21931 rewires.
  • Module-level comment attachment (L10 cure): nearest-following-declaration attribution for comments outside function bodies so //! and top-level // comments carry ON_SYMBOL edges
  • Reactivate + verify the comment detectors (scripts/impl_hygiene_review_runtime/seam_scan/comment_detectors.py) against full-compiler comment coverage: (a) rewire the detectors’ staleness gate (comment_detectors.py:50) to ALSO read the comments-facet staleness (comments_wide_stale) ALONGSIDE the existing shared generic cpg_stale — never narrow the shared gate, which also serves CFG/DDG, typed-dataflow, and attribute detectors; (b) reconcile the ‘6 detectors’ count against the extractor’s 6 quality props — is_ephemeral_ref currently has no dedicated detector function at HEAD — by adding its detector or explicitly documenting the exclusion; (c) re-measure the coverage census (crate distribution, attach-rate, flagged-candidate volume including the orphaned-but-flagged set) AFTER both w-1b578988 and w-4b4ffdb0 land, since the pre-cure counts are a floor, not the target; (d) drive the ori-comment-node-presence fitness probe (intel_repo/queries/fitness/probe-corpus.json:75) green without weakening it to a tautology or #skip-ing detector tests; tests.

Fresh intel (regenerated)

This section extends the intel graph’s OWN comment-ingestion facet, and the graph cannot see the code it is about - intel_repo/extractor/cpg/src/comments.rs and scripts/impl_hygiene_review_runtime/seam_scan/comment_detectors.py return ZERO symbols (the “ori” repo indexes compiler/ only). The code-graph tiers (CALLS / similar / callers / MISSING_ABSTRACTION) are therefore DARK for the terrain. BUT the extractor’s OUTPUT - 13,477 :CodeComment nodes - is fully queryable, and it exposes the section’s three work items as concrete, measurable defects.

The single decisive [JOIN]: the ON_SYMBOL attach model and the L10 cure are the same defect measured two ways. ON_SYMBOL is emitted only from a comment’s enclosing function-BODY CfgBlock (owning_block matches kind IN ['body','closure-body'] only). Doc comments (///, //!) sit ABOVE items/modules, never inside a body - so all 8,390 doc comments carry ZERO ON_SYMBOL edges and every quality-prop the six detectors read is invisible for them (the detectors JOIN (:CodeComment)-[:ON_SYMBOL]->(:Symbol)). Only ~27% of comments (3,613 / 13,477) attach at all. That is not a coverage nicety - it is the L10 cure (w-4b4ffdb0) and it is what unblocks the detector reactivation (w-d0c21931).

Decisions this package forces before any code is written:

  1. Widen the DRIVER, not the CFG/DDG scope. The comments-facet scope knob CPG_SCOPE_PREFIXES_BINDER_COMMENTS is ALREADY [] (repo-wide); yet 96% of nodes (12,978 / 13,477) are ori_arc because the extractor is only INVOKED on the aims CFG/DDG scope + the 4 §14 subsystems. Widening (w-1b578988) means running the comments pass on the full compiler - NEVER touching CPG_SCOPE_PREFIXES (that is the aims-scoped CFG/DDG gate whose budget the §08.4 protocol caps).
  2. The L10 cure must add a new attachment path, not reuse owning_block. Attaching module/item docs requires nearest-FOLLOWING-declaration attribution (a :Symbol), a structurally different lookup from the enclosing-body containment owning_block does today.
  3. Give the comments facet its own staleness stamp. comments_wide_stale is a registered facet (graph_contract.py:95) with a null last_comments_wide_at. The six detectors gate on the generic cpg_stale today (comment_detectors.py:50 -> cypher_query.cpg_stale); w-d0c21931 rewires them to the comments-facet reading. Do NOT narrow the shared generic cpg_stale - it also gates CFG/DDG, typed-dataflow, and attribute detectors.
  4. Reactivation is gated by the fitness probe ori-comment-node-presence (intel_repo/queries/fitness/probe-corpus.json:75), today a weak count > 0 presence check; the north-star gate (scripts/intel-query.sh fitness) reads it green.
["CPG_SCOPE_PREFIXES", "cpg_stale", "_cpg_stale_via_changed_files", "cpg_extraction_fingerprint"]

DIG DEEPER

scripts/intel-query.sh dag-ascii intel-graph-deepening
scripts/intel-query.sh file-symbols intel_repo/extractor/cpg/src/comments.rs --repo ori   # returns 0 - self-referential blind spot
scripts/intel-query.sh cypher "MATCH (c:CodeComment {repo:'ori'}) OPTIONAL MATCH (c)-[r:ON_SYMBOL]->() RETURN count(c) AS total, count(r) AS attached" --human

(full dossier: comment-ingestion-repo-wide—s-916f0932.intel.md)