memory: fix extraction trigger, dedupe, and add runtime diagnostics

Four defects found by end-to-end testing, not by reading the code.

1. Extraction never ran, silently. Collecting every event type that
   ctx.event.subscribe actually delivers shows `session.idle` is NOT among
   them — it is defined in the protocol but never emitted on this stream.
   The subscription filtered on it alone, so auto-extraction was a permanent
   no-op with no error. Now debounced off events that do arrive
   (session.step.started / session.tool.called / session.inbox.delivered)
   after `settleMs` of quiet, which is what "idle" meant anyway. Verified:
   the trigger now fires and reports "轮数不足" for a 1-turn session.

2. Dedupe silently stopped rejecting duplicates. It compared against a fixed
   retrieval score, but that score carries idf and scope priors that drift as
   the corpus grows, so an obvious restatement of an existing memory scored
   0.48 against a 0.9 threshold and was written as a rival entry. Now uses
   term coverage of the candidate, which answers "is this already said?" and
   does not move with corpus size. Verified: a 100%-duplicate is rejected, a
   novel pitfall is kept.

3. Harness-injected user turns became the retrieval query. "<system-reminder>"
   and "The server restarted while you were working" are user-role messages,
   so the hook retrieved on them and injected memories chosen by noise. Both
   are filtered out now.

4. The `pinned` flag was absent from the remember tool's input schema, so the
   schema stripped it and pinning was unreachable from the tool surface.
   `kind` also ignored the rules/ category when a stale value was stored.

memory_inspect now reports runtime health for the two paths with no other
observable effect: last injection size and query, and last extraction outcome
with its reason. Without it, "working" and "silently dead" look identical —
which is exactly how defects 1 and 2 survived the first round of testing.
This commit is contained in:
2026-10-09 10:28:34 +08:00
parent a64ea2acb3
commit 92c612b94c
5 changed files with 242 additions and 39 deletions
+43 -8
View File
@@ -13,7 +13,7 @@
// would let a hallucination become permanent policy.
import type { Context } from "@opencode/plugin"
import { scoreFiles } from "./recall.ts"
import { tokenize } from "./recall.ts"
export interface ExtractOptions {
enabled: boolean
@@ -222,14 +222,20 @@ export interface ExtractionOutcome {
/**
* Decide which candidates are worth writing, using existing memories for
* dedupe. Near-duplicates bump the existing file instead of adding a rival.
* dedupe.
*
* Dedupe uses term COVERAGE of the candidate, not the retrieval score. The
* score carries idf and scope priors that shift as the corpus grows, so a fixed
* score threshold silently stops rejecting duplicates as the library fills up.
* Coverage answers the question we actually care about: "does an existing
* memory already say this?"
*/
export function screenCandidates(
candidates: Candidate[],
existing: readonly import("./store.ts").MemoryFile[],
options: { minimumScore?: number } = {},
options: { coverage?: number } = {},
): ExtractionOutcome {
const threshold = options.minimumScore ?? 0.9
const threshold = options.coverage ?? 0.55
const accepted: Candidate[] = []
const rejected: { candidate: Candidate; reason: string }[] = []
@@ -246,11 +252,14 @@ export function screenCandidates(
candidate.category = "rules"
candidate.kind = "rule-proposal"
}
const probe = `${candidate.title}\n${candidate.summary}\n${candidate.body}`
const scored = scoreFiles(existing, probe, {}, { halfLifeDays: 0, pinnedBoost: false })
const top = scored[0]
if (top && top.score >= threshold) {
rejected.push({ candidate, reason: `与已有记忆高度重叠 (${top.file.rel}, score ${top.score.toFixed(2)})` })
const best = bestCoverage(probe, existing)
if (best.file && best.coverage >= threshold) {
rejected.push({
candidate,
reason: `与已有记忆重复 ${(best.coverage * 100).toFixed(0)}% (${best.file.rel})`,
})
continue
}
accepted.push(candidate)
@@ -259,6 +268,32 @@ export function screenCandidates(
return { candidates: accepted, rejected }
}
/**
* Fraction of the candidate's distinctive terms that an existing memory also
* contains. The candidate's own terms are the query; the best match wins.
*/
function bestCoverage(
probe: string,
existing: readonly import("./store.ts").MemoryFile[],
): { file?: import("./store.ts").MemoryFile; coverage: number } {
const probeTerms = new Set(tokenize(probe).filter((t) => t.length >= 2))
if (probeTerms.size === 0) return { coverage: 0 }
let best: { file?: import("./store.ts").MemoryFile; coverage: number } = { coverage: 0 }
for (const file of existing) {
if (file.fm.superseded_by) continue
// Same slug in the same root is the same memory, whatever the wording.
const doc = new Set(
tokenize([file.title, file.summary, file.highlight, file.body].join("\n")).filter((t) => t.length >= 2),
)
let shared = 0
for (const term of probeTerms) if (doc.has(term)) shared++
const coverage = shared / probeTerms.size
if (coverage > best.coverage) best = { file, coverage }
}
return best
}
export interface RunExtractionInput {
ctx: Context
model: { providerID: string; id: string }