Repository navigation
fix(sections): stop get_section's retry from reusing the full graph's search index (#5385) - #5386
Deepjyoti-Sarmah wants to merge 6 commits into
Conversation
|
Thanks for the pull request, @Deepjyoti-Sarmah. A maintainer will review it soon. Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions. A couple of things that speed up review: make sure the test suite passes on Python 3.11 and 3.13, and that the change keeps extraction deterministic. |
There was a problem hiding this comment.
Graphify review 
1 bug · 1 possible issue to check · High impact
The section resolver retries failed lookups against a graph view containing only Markdown section nodes.
Non-string graph metadata keys can crash section lookup ·
sections.py:196
1 possible issue we couldn't confirm
- The cache-isolation check on line 70 passes even when the section view has no trigram index at all ·
tests/test_section_subgraph_index.py:70
They don't change the score.
Code graph · 379 functions depend on this change
- 4 of 614 test files are affected by this change.
- Gate, test list and every flagged function: in the Graphify check.
Reviewed 8c7af1d · 3 earlier findings not re-checked · Full report in the Graphify check
…raphify-Labs#5385) Behaviour-neutral: the constant is not used yet.
…hify-Labs#5385) G.subgraph() shares G.graph, so the get_section retry reused the full graph's trigram index, whose ids include nodes outside the view, and _find_node_tiers raised KeyError. The view now gets its own graph dict without the derived search caches.
d487198 to
0d7b544
Compare
| # caches (ids of nodes not in the view) or store its own over the full | ||
| # graph's. Keep public graph metadata and the logical direction flag, but | ||
| # discard other private state so future caches cannot leak into this view. | ||
| sections.graph = { |
There was a problem hiding this comment.
Non-string graph metadata keys can crash section lookup
A section lookup that needs the retry can raise AttributeError when the graph has a non-string metadata key.
Check that graph metadata keys are strings before calling startswith in resolve_section_query in graphify/sections.py.
Evidence · checked against the code
Expected: The resolver returns its normal section match or not-found result.
Actually: The retry’s metadata filter calls startswith on the non-string key and raises AttributeError.
- Independent source review: The CLI caller loads a graph before calling the resolver, and the MCP caller uses its graph; these paths do not establish that all callers supply JSON-loaded graphs. The inspected tests cover retry behavior and cache isolation but do not test non-string graph metadata keys.
- The inspected CLI and MCP call sites use graphs loaded from JSON, whose metadata keys are strings; the issue applies to callers that supply a NetworkX graph with a non-string metadata key.
- Also reported as: Section lookup can raise an exception when the graph's metadata has a non-text key
Prompt to fix with AI
In graphify/sections.py, resolve_section_query (lines 193-196):
Problem: Non-string graph metadata keys can crash section lookup
Expected: The resolver returns its normal section match or not-found result.
Actually: The retry’s metadata filter calls `startswith` on the non-string key and raises `AttributeError`.
Fix it at the cause, and add a test that fails without the fix.
Graphify · Checked against the code · View in Graphify
What does this PR do?
Fixes #5385.
get_sectionraisesKeyError: 'Key deploy_sh not found'when thequery is ambiguous or resolves to a non-section node.
resolve_section_queryresolves in two passes:A networkx subgraph view is not a new graph. It shares the parent's
graphdict, and that is where
serve.pycaches its search state. The retry thereforereads the full graph's trigram index. Its
idsincludedeploy_sh, which is notin the view, and
_find_node_tiersdoesG.nodes[nid]on it.The view keeps every public key, plus the private keys a lookup reads. That is
_SECTION_VIEW_GRAPH_KEYS = {"_logical_directed"}. Every other_-prefixed keyis dropped. That covers the four search caches
serve.pykeeps today(
_trigram_index,_idf_cache,_traversal_view,_path_search_graphs) andany cache added later.
serve.pyalso stores_learning_overlayand_built_at_commit. Neither_resolve_single_nodenor its callees read them, so dropping them from thistemporary view doesn't change a lookup.
It is not only the warm case
The issue says the index has to be warm, which the MCP server does at load
(
serve.py:167). It does not have to be. The first pass,_resolve_single_node(G, query), builds the full graph's index itself. So anyquery that falls through to the retry crashes, on a cold graph too. The test
parametrizes both.
The sharing is also unsafe in the other direction, though this function does not
reach it today: a view searched before its parent would write a sections-only
index into the shared dict, and the full graph's later searches would miss code
nodes.
test_retry_does_not_replace_full_graph_indexchecks that the fullgraph's index still covers every node after a retry.
Why not
.copy()The issue suggests
G.subgraph(...).copy(). That alone does not fix it: networkxcopies
G.graphinto the new graph, caches included. I checked:It would also copy every section node and edge on each fallback query. A fresh
dict on the view costs one small dict comprehension.
The other suggestion, filtering candidates with
nid in Gin_trigram_candidates, would stop the crash. But the view would still shareevery other cache with the parent. Fixing the shared dict covers all of them.
Five commits
The last two answer the review.
065464a7proves the retry runs on anisolated view, and
563b5db8turns the explicit cache list into an allowlist,so a cache added to
G.graphlater is not shared by default. Each partreverts alone: reverting
563b5db8goes back to the explicit list, and theunknown-cache test fails again.
Type of change
Verification & Invariants
The invariant: a search on a graph view uses an index built from that view's
nodes, and never changes the parent graph's caches.
Merge Danger
Door: two-way. Nothing persisted changes: no schema, cache-key or file-format
change. Only the in-memory view used for the retry is touched.
Blast radius: small.
and are unaffected.
builds a sections-only index. That costs work proportional to the number of
section nodes, on the fallback path only. On a synthetic graph of 2,186 nodes
(the largest checked-in example graph plus 300 section nodes), the median of
9 fresh builds was about 10 ms for the sections view against about 120 ms
for the full graph. That is too small to justify a cache keyed on the parent,
with its own invalidation and lifetime, so I left it out.
Limitations
starts reading later has to be added to
_SECTION_VIEW_GRAPH_KEYS. Missing itcan only make the view miss metadata. It can't make it reuse a full-graph cache.
G.subgraph(...)uses (cluster.py) do not call theserve.pysearchhelpers, so they are not affected. I did not change them.
How was this tested?
tests/test_section_subgraph_index.pyis new and has 6 tests, built on the issue'ssynthetic graph:
test_non_section_query_does_not_raise[False]and[True]: the issue's query,cold and warm, gives the same answer as a fresh graph.
test_retry_does_not_replace_full_graph_index: after a retry, the full graph'sindex still covers every node, and
filler 7still resolves.test_heading_still_resolves_with_warm_index: a valid heading still resolves onthe first pass.
test_retry_uses_isolated_section_view: a spy on_resolve_single_nodechecksthat the retry runs on exactly the section nodes, with its own
.graphthatkeeps
_logical_directed. It also checks that, with the full index warm, theview never holds the full graph's
_trigram_index.test_retry_drops_unknown_future_cache: a key the code doesn't know,_some_new_cache, is not visible on the view.The first three fail on v8 with the issue's
KeyError, and so do the two newones. All six pass here. On
0d7b5440, before the allowlist,test_retry_drops_unknown_future_cachefails. The issue's repro now prints thenormal message:
All with
PYTHONPATHset to the worktree:Not tested: Windows. Nothing here builds paths.
Graphify-specific checklist
uv run python -m tools.skillgen --bless) when changing their source fragments. (N/A: no skill fragments touched.)