Skip to content
This repository was archived by the owner on Aug 24, 2026. It is now read-only.

Commit 535725b

Browse files
groksrcclaude
andcommitted
fix: surface result titles in QA context assembly
LoCoMo v2 q300 exposed a structural asymmetry: the session-date fix lifted multi_hop to 0.57 (mem0) and 0.52 (grep) but bm-local stayed at 0.06. BM returns bullet-level matched chunks as hit text — precise, but they strip the document context where the session date lives. grep returns doc heads (date included) and mem0 stores whole documents, so only BM lost temporal anchoring. BM's search results DO return the title (which carries the date); the provider now passes it through hit metadata and context assembly adds it to the section header when it isn't already in the snippet. This uses only what the provider returns — no special-casing. The underlying product gap (BM chunks lack document-level context at retrieval time) is queued as the first basic-memory Phase 2 change. 2 new tests; suite green, lint clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Drew Cain <groksrc@gmail.com>
1 parent b60bd2b commit 535725b

2 files changed

Lines changed: 33 additions & 1 deletion

File tree

src/basic_memory_benchmarks/scoring/qa.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -140,7 +140,11 @@ def assemble_context(hits: list[SearchHit], max_chars: int = CONTEXT_MAX_CHARS)
140140
if not snippet:
141141
break
142142
source = hit.source_doc_id or hit.source_path or "unknown"
143-
sections.append(f"[Memory {rank} | source: {source}]\n{snippet}")
143+
title = (hit.metadata or {}).get("title")
144+
header = f"[Memory {rank} | source: {source}]"
145+
if title and str(title) not in snippet:
146+
header = f"[Memory {rank} | source: {source} | {title}]"
147+
sections.append(f"{header}\n{snippet}")
144148
used += len(snippet)
145149
if used >= max_chars:
146150
break

tests/test_qa_scoring.py

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -350,3 +350,31 @@ def test_budget_override_flows_through_run_qa(self):
350350
max_context_chars=30_000,
351351
)
352352
assert answerer.prompts[0].count("z") == 30_000
353+
354+
355+
class TestHitTitleInContext:
356+
def test_title_metadata_lands_in_header(self):
357+
from basic_memory_benchmarks.models import SearchHit
358+
from basic_memory_benchmarks.scoring.qa import assemble_context
359+
360+
hit = SearchHit(
361+
source_doc_id="doc-a",
362+
text="- **Melanie:** I hiked yesterday!",
363+
score=1.0,
364+
metadata={"title": "locomo-c00-s18 (3:01 pm on 20 October, 2023)"},
365+
)
366+
ctx = assemble_context([hit])
367+
assert "| locomo-c00-s18 (3:01 pm on 20 October, 2023)]" in ctx
368+
369+
def test_title_skipped_when_already_in_text(self):
370+
from basic_memory_benchmarks.models import SearchHit
371+
from basic_memory_benchmarks.scoring.qa import assemble_context
372+
373+
hit = SearchHit(
374+
source_doc_id="doc-a",
375+
text="# Chat session at 8 May 2023\nfull body",
376+
score=1.0,
377+
metadata={"title": "Chat session at 8 May 2023"},
378+
)
379+
ctx = assemble_context([hit])
380+
assert ctx.count("Chat session at 8 May 2023") == 1

0 commit comments

Comments
 (0)