100%

Incremental-sync redesign

Goal

Keep per-commit sync O(changes) with correct cross-file invalidation, extending the existing run_incremental two-phase pipeline (intel_repo/neo4j/sync_ori_graph.py). That pipeline already propagates per-file SHA256 content_hash (sync_ori_graph.py:418, hashlib.sha256) and import_code_graph.py already does unit DETACH DELETE + content_hash storage; the redesign adds the skip-on-unchanged gate, the unit=file stamp, late-bound CALLS resolution, and fanout-only re-resolution on top of that machinery, never a parallel sync path.

SCIP-precise edge emission moved out of this plan: the former scip-from-oric section (s-61848b41) is superseded and the SCIP-from-oric work now lives in plans/completed/scip-emission. This section depends on neither — incremental re-sync preserves whatever CALLS edges the graph already carries (tree-sitter or compiler-resolved provenance) and re-resolves them by late-bound symbol string, so it is provenance-agnostic.

Implementation Sketch

Each work item is a gap-fill on the existing run_incremental two-phase path, not a greenfield sync:

  • Content-hash gate (w-0bef982e): the per-file SHA256 content_hash is already computed and stored; add the comparison that skips a file whose stored hash equals the recomputed hash, so unchanged units never re-enter phase 2.
  • Unit-per-file invalidation (w-60e68bb0): DETACH DELETE already exists; stamp every node unit=file so a changed file’s nodes are deleted and recreated as one unit.
  • Late-bound CALLS resolution (w-de11b5a8): resolve CALLS by symbol string (ref.symbol == def.symbol) at re-resolution time, never as a frozen endpoint-pair edge — so re-resolving a renamed/moved symbol re-binds correctly. This preserves the arity- + kind-filtered symbol-string resolution invariant established by calls-resolution-precision (s-1209f1e9): the homonym false-binds removed there (e.g. ori_parse::cursor::expect) must not regress on re-sync; a re-resolved CALLS edge stays kind-filtered to callable targets and arity-compatible.
  • Fanout-only re-resolution + export-set early-cutoff (w-ac78ad13): re-resolve only the symbol fanout (refs in X plus incoming refs to X’s symbols); when X’s exported signature set is byte-identical, early-cutoff its dependents (Glean / stack-graphs / Salsa incrementality pattern). The correctness hazard lives here: a dependent file with unchanged bytes whose upstream export set changed MUST still be re-resolved (the stale-miss case the positive stale-miss pin guards; the byte-identical-export early-cutoff is the negative pin).

Work Items

  • Content-hash file gate: SHA256 each file; skip unchanged units on re-sync.
  • Unit-per-file node invalidation (stamp every node unit=file; DETACH DELETE changed units, recreate).
  • Resolve CALLS by late-bound symbol string (ref.symbol == def.symbol), never a frozen endpoint-pair edge.
  • Fanout-only re-resolution (refs in X plus incoming refs to X’s symbols) + export-set early-cutoff (skip dependents when X’s exported signature set is unchanged).

Test Coverage

This is an intel_repo Python/Neo4j slice with no compiler-runtime surface, so the L1-L11 pipeline matrix (parse / typecheck / canonicalize / ARC / eval / LLVM-debug / LLVM-release / AOT / dual-exec-parity / leak / spec) is N/A: there is no .ori program executed, no IR emitted, and no RC to balance — the deliverable is graph state and sync behavior. Verification is the intel_repo pytest gate plus an automated O(fanout) regression pin AND an L12 production-entry test driving the real sync CLI, matching the bar set by the sibling intel_repo sections (s-1209f1e9 calls-resolution-precision, s-1c2551f0 cpg-dataflow), which each pin their slice with a named intel_repo/tests/test_*.py pytest carrying positive + negative pins AND an L12 production-entry criterion against the real entry point.

  • pytest gate: intel_repo/.venv/bin/python -m pytest intel_repo/tests/test_incremental_sync.py — asserts, against a seeded live-graph fixture: (positive) a content-unchanged file is skipped on re-sync (no node delete/recreate); (positive) a content-changed file’s unit is DETACH-DELETEd and recreated stamped unit=file; (positive stale-miss) an unchanged-bytes dependent whose upstream exported-signature set changed is re-resolved; (negative early-cutoff) an upstream whose exported-signature set is byte-identical does NOT trigger its dependents’ re-resolution; (negative homonym) a re-resolved CALLS edge stays kind-filtered to a callable, arity-compatible target and does not false-bind a ::cursor::expect-style homonym.
  • automated O(fanout) regression pin (sc-f772d4e7): intel_repo/.venv/bin/python -m pytest intel_repo/tests/test_incremental_sync.py -k fanout_bound — re-syncs a 1-file change on a seeded live graph and asserts the re-resolved-unit count stays at or below the changed symbol’s fanout-set size (documented acceptance threshold), NOT the full repo unit count. This is an automated regression gate with a stated bound, NOT a manual stopwatch.
  • L12 production-entry probe (sc-43d1e7b5, work-item-bound to w-ac78ad13): drive the REAL incremental sync path (intel_repo/scripts/build-code-graph.sh / sync-ori-graph.sh --changed → run_incremental) on a real single-file edit against the live ori graph; assert fanout-only re-resolution AND that a real stale-miss dependent (unchanged bytes, changed upstream export-set) is re-resolved end-to-end. Driven through the real CLI entry point, NOT a seeded-fixture unit test — mirrors the sibling cpg-dataflow L12 criterion (sc-d1f7b5).
  • stale-miss correctness probe (sc-43d1c0c1, work-item-bound to w-ac78ad13): intel_repo/.venv/bin/python -m pytest intel_repo/tests/test_incremental_sync.py -k stale_miss covers the export-set early-cutoff stale-miss positive + negative pins above.

References

intel_repo/neo4j/sync_ori_graph.py (run_incremental, content_hash at line 418); intel_repo/neo4j/import_code_graph.py (unit DETACH DELETE + content_hash storage); intel_repo/scripts/build-code-graph.sh; calls-resolution-precision (s-1209f1e9, arity+kind-filtered symbol-string resolution); Glean / stack-graphs / Salsa incrementality.