Add memory plugin: three-tier markdown store with plugin-driven retrieval
A durable memory layer for OpenCode. Structure:
ROOT.md total plan: goals, hard rules, directory registry,
and a scenario -> file routing table
<category>/
index.md one line per file, COMPILED from frontmatter
<slug>.md the memory itself
Two roots, same shape: <config>/memory/ (global) and
<project canonical>/.memory/ (project). Project wins on conflict; global
hard rules stay resident everywhere.
Why a plugin rather than a skill that tells the model to read a folder:
model-driven retrieval is unreliable — the model forgets, reads half the
files, or reads the wrong ones. Markdown is the human-facing format; the
plugin is the engine that parses, scores, and injects.
Measured, not assumed: FTS5's two built-in tokenizers both fail on this
corpus. On SQLite 3.53.4 with "部署脚本用 pnpm 而不是 npm...": unicode61
emits 7 terms for the whole sentence and matches 0 hits for "部署";
trigram needs >= 3 characters and also returns 0 for that two-character
word. So tokenization happens in the plugin (CJK bigrams, latin words)
and no SQL index is used — a thousand memories are fine to scan.
index.md and ROOT's two tables are compiled, never hand-edited: a
hand-maintained index drifts, a generated one cannot. Hit counters live
in .state.json so cross-device sync does not conflict on every recall.
Tools are named recall/remember/forget/inspect rather than
write/read/status: generic names were observed shadowing the harness's
own built-in tools.
This commit is contained in:
@@ -0,0 +1,289 @@
|
||||
// Automatic extraction: turn a finished turn into candidate memories.
|
||||
//
|
||||
// Runs on `session.idle`. The point is that memory capture must not depend on
|
||||
// the model remembering to call `memory_write` — models forget. This is the
|
||||
// safety net.
|
||||
//
|
||||
// Two deliberate limits keep it from becoming a garbage generator:
|
||||
//
|
||||
// 1. A separate, cheap model does the extraction. It never touches the main
|
||||
// conversation, so a bad extraction cannot derail the user's work.
|
||||
// 2. Hard rules are never written automatically. The extractor may *propose*
|
||||
// one, but only a human confirms it. Letting a model write "iron rules"
|
||||
// would let a hallucination become permanent policy.
|
||||
|
||||
import type { Context } from "@opencode/plugin"
|
||||
import { scoreFiles } from "./recall.ts"
|
||||
|
||||
export interface ExtractOptions {
|
||||
enabled: boolean
|
||||
/** Minimum turns in a session before extraction kicks in. */
|
||||
minTurns: number
|
||||
/** Cap on how many messages are handed to the extractor. */
|
||||
maxMessages: number
|
||||
model?: { providerID: string; id: string }
|
||||
}
|
||||
|
||||
export const DEFAULT_EXTRACT: ExtractOptions = {
|
||||
enabled: true,
|
||||
minTurns: 4,
|
||||
maxMessages: 24,
|
||||
}
|
||||
|
||||
export interface Candidate {
|
||||
category: string
|
||||
slug: string
|
||||
title: string
|
||||
summary: string
|
||||
highlight?: string
|
||||
body: string
|
||||
kind?: string
|
||||
loadWhen?: string[]
|
||||
/** Rules the model believes should be hard rules, needing human confirmation. */
|
||||
proposeRule?: boolean
|
||||
}
|
||||
|
||||
const SYSTEM = `You extract durable memories from a coding conversation.
|
||||
|
||||
Return ONLY a JSON array. Each element:
|
||||
{
|
||||
"category": "preferences|decisions|pitfalls|facts",
|
||||
"slug": "kebab-case-identifier",
|
||||
"title": "short title",
|
||||
"summary": "ONE sentence, under 100 chars. This is what shows in an index.",
|
||||
"highlight": "the single most important detail",
|
||||
"body": "markdown detail, 2-6 sentences",
|
||||
"loadWhen": ["trigger words or situations where this is relevant"],
|
||||
"proposeRule": false
|
||||
}
|
||||
|
||||
Only record what stays true after the session ends:
|
||||
GOOD A stated preference, an architectural decision and its reason,
|
||||
a non-obvious gotcha and its root cause, a project convention.
|
||||
BAD What the code currently looks like, task state, git branch names,
|
||||
file paths, anything already obvious from the repo, one-off chatter.
|
||||
|
||||
NEVER set proposeRule to true unless the user stated an absolute rule about how
|
||||
to behave ("always", "never", "必须", "禁止"). Doubtful -> omit the memory.
|
||||
|
||||
If nothing durable was learned, return [].`
|
||||
|
||||
/**
|
||||
* Pull plain text out of a session message without depending on the exact
|
||||
* message shape: the plugin package ships types for a schema the host may
|
||||
* evolve, so this walks defensively and collects `text` fields.
|
||||
*/
|
||||
export function collectText(value: unknown, depth = 0, out: string[] = []): string[] {
|
||||
if (depth > 6 || out.length > 400) return out
|
||||
if (typeof value === "string") {
|
||||
if (value.trim()) out.push(value)
|
||||
return out
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
for (const item of value) collectText(item, depth + 1, out)
|
||||
return out
|
||||
}
|
||||
if (value && typeof value === "object") {
|
||||
const record = value as Record<string, unknown>
|
||||
// Prefer a part's own text over its metadata noise.
|
||||
if (typeof record.text === "string" && record.text.trim()) out.push(record.text)
|
||||
for (const [key, child] of Object.entries(record)) {
|
||||
if (key === "text") continue
|
||||
if (key === "synthetic" || key === "cache" || key === "providerMetadata") continue
|
||||
collectText(child, depth + 1, out)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
export function userAndAssistantText(messages: readonly unknown[]): string {
|
||||
const parts: string[] = []
|
||||
for (const message of messages) {
|
||||
const record = message as Record<string, unknown> | null
|
||||
if (!record) continue
|
||||
const role = record.role
|
||||
if (role !== "user" && role !== "assistant") continue
|
||||
const texts = collectText(record.content)
|
||||
const joined = texts.join("\n").trim()
|
||||
if (joined) parts.push(`${role === "user" ? "USER" : "ASSISTANT"}: ${joined}`)
|
||||
}
|
||||
return parts.join("\n\n")
|
||||
}
|
||||
|
||||
/** `generate.text` is documented as returning text; accept a few shapes. */
|
||||
function coerceText(result: unknown): string {
|
||||
if (typeof result === "string") return result
|
||||
if (result && typeof result === "object") {
|
||||
const record = result as Record<string, unknown>
|
||||
for (const key of ["text", "content", "output", "result"]) {
|
||||
const value = record[key]
|
||||
if (typeof value === "string") return value
|
||||
if (Array.isArray(value)) {
|
||||
const joined = collectText(value).join("\n")
|
||||
if (joined) return joined
|
||||
}
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
/** Parse the extractor's JSON array, tolerating prose and code fences. */
|
||||
export function parseCandidates(raw: string): Candidate[] {
|
||||
const text = raw.trim()
|
||||
if (!text) return []
|
||||
const fenced = /```(?:json)?\s*([\s\S]*?)```/.exec(text)
|
||||
const body = fenced?.[1] ?? text
|
||||
const start = body.indexOf("[")
|
||||
const end = body.lastIndexOf("]")
|
||||
if (start < 0 || end <= start) return []
|
||||
|
||||
let parsed: unknown
|
||||
try {
|
||||
parsed = JSON.parse(body.slice(start, end + 1))
|
||||
} catch {
|
||||
return []
|
||||
}
|
||||
if (!Array.isArray(parsed)) return []
|
||||
|
||||
const out: Candidate[] = []
|
||||
for (const entry of parsed) {
|
||||
if (!entry || typeof entry !== "object") continue
|
||||
const record = entry as Record<string, unknown>
|
||||
const title = str(record.title)
|
||||
const bodyText = str(record.body)
|
||||
if (!title || !bodyText) continue
|
||||
|
||||
const category = str(record.category).toLowerCase() || "facts"
|
||||
const slug =
|
||||
slugify(str(record.slug)) || slugify(title)
|
||||
if (!slug) continue
|
||||
|
||||
out.push({
|
||||
category: category.replace(/[^a-z0-9._-]/g, "") || "facts",
|
||||
slug,
|
||||
title,
|
||||
summary: str(record.summary) || title,
|
||||
highlight: str(record.highlight) || undefined,
|
||||
body: bodyText,
|
||||
kind: str(record.kind) || undefined,
|
||||
loadWhen: Array.isArray(record.loadWhen)
|
||||
? record.loadWhen.map((v) => str(v)).filter(Boolean).slice(0, 8)
|
||||
: undefined,
|
||||
proposeRule: record.proposeRule === true,
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function str(value: unknown): string {
|
||||
return typeof value === "string" ? value.trim() : ""
|
||||
}
|
||||
|
||||
export function slugify(input: string): string {
|
||||
const ascii = input
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, "-")
|
||||
.replace(/^-+|-+$/g, "")
|
||||
.slice(0, 48)
|
||||
if (ascii) return ascii
|
||||
// CJK titles have no ASCII to work with; fall back to a hash-free stable id.
|
||||
return [...input].map((ch) => ch.codePointAt(0)!.toString(36)).join("").slice(0, 24)
|
||||
}
|
||||
|
||||
/** Secrets that must never reach a git-tracked memory file. */
|
||||
const SECRET_PATTERNS: RegExp[] = [
|
||||
/\b(?:sk|pk|rk)-[A-Za-z0-9_-]{16,}/g,
|
||||
/\bgh[pousr]_[A-Za-z0-9]{16,}/g,
|
||||
/\bAKIA[0-9A-Z]{16}\b/g,
|
||||
/\bxox[baprs]-[A-Za-z0-9-]{10,}/g,
|
||||
/\bey[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b/g,
|
||||
/\b(?:password|passwd|secret|api[_-]?key|access[_-]?token|auth[_-]?token)\s*[:=]\s*\S+/gi,
|
||||
/-----BEGIN[A-Z ]*PRIVATE KEY-----[\s\S]*?-----END[A-Z ]*PRIVATE KEY-----/g,
|
||||
]
|
||||
|
||||
export function hasSecret(text: string): string | undefined {
|
||||
for (const pattern of SECRET_PATTERNS) {
|
||||
const match = pattern.exec(text)
|
||||
if (match) return match[0].slice(0, 24)
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
export function redactSecrets(text: string): string {
|
||||
let out = text
|
||||
for (const pattern of SECRET_PATTERNS) out = out.replace(pattern, "[REDACTED]")
|
||||
return out
|
||||
}
|
||||
|
||||
export interface ExtractionOutcome {
|
||||
candidates: Candidate[]
|
||||
rejected: { candidate: Candidate; reason: string }[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide which candidates are worth writing, using existing memories for
|
||||
* dedupe. Near-duplicates bump the existing file instead of adding a rival.
|
||||
*/
|
||||
export function screenCandidates(
|
||||
candidates: Candidate[],
|
||||
existing: readonly import("./store.ts").MemoryFile[],
|
||||
options: { minimumScore?: number } = {},
|
||||
): ExtractionOutcome {
|
||||
const threshold = options.minimumScore ?? 0.9
|
||||
const accepted: Candidate[] = []
|
||||
const rejected: { candidate: Candidate; reason: string }[] = []
|
||||
|
||||
for (const candidate of candidates) {
|
||||
const secret = hasSecret(`${candidate.title}\n${candidate.summary}\n${candidate.body}`)
|
||||
if (secret) {
|
||||
rejected.push({ candidate, reason: `含疑似密钥 (${secret}…)` })
|
||||
continue
|
||||
}
|
||||
if (candidate.proposeRule) {
|
||||
// Not rejected outright — a proposal is surfaced for a human, which is
|
||||
// handled by the caller. Flag it here so it is written as a normal
|
||||
// memory under `proposals` rather than promoted to policy.
|
||||
candidate.category = "rules"
|
||||
candidate.kind = "rule-proposal"
|
||||
}
|
||||
const probe = `${candidate.title}\n${candidate.summary}\n${candidate.body}`
|
||||
const scored = scoreFiles(existing, probe, {}, { halfLifeDays: 0, pinnedBoost: false })
|
||||
const top = scored[0]
|
||||
if (top && top.score >= threshold) {
|
||||
rejected.push({ candidate, reason: `与已有记忆高度重叠 (${top.file.rel}, score ${top.score.toFixed(2)})` })
|
||||
continue
|
||||
}
|
||||
accepted.push(candidate)
|
||||
}
|
||||
|
||||
return { candidates: accepted, rejected }
|
||||
}
|
||||
|
||||
export interface RunExtractionInput {
|
||||
ctx: Context
|
||||
model: { providerID: string; id: string }
|
||||
transcript: string
|
||||
existing: readonly import("./store.ts").MemoryFile[]
|
||||
signal?: AbortSignal
|
||||
}
|
||||
|
||||
/** One model round-trip. Returns screened candidates and why things were dropped. */
|
||||
export async function extract(input: RunExtractionInput): Promise<ExtractionOutcome> {
|
||||
const prompt = `${SYSTEM}\n\n---\n\n${input.transcript}`
|
||||
let result: unknown
|
||||
try {
|
||||
// `generate.text` has no cancellation documented on this surface, so the
|
||||
// signal is forwarded when accepted and simply ignored when not.
|
||||
result = await (input.ctx.generate as { text: (input: unknown) => Promise<unknown> }).text({
|
||||
model: input.model,
|
||||
prompt,
|
||||
...(input.signal ? { signal: input.signal } : {}),
|
||||
})
|
||||
} catch {
|
||||
// An extraction failure must never disturb the user's session.
|
||||
return { candidates: [], rejected: [] }
|
||||
}
|
||||
const text = coerceText(result)
|
||||
const candidates = parseCandidates(text)
|
||||
return screenCandidates(candidates, input.existing)
|
||||
}
|
||||
Reference in New Issue
Block a user