Add memory plugin: three-tier markdown store with plugin-driven retrieval
A durable memory layer for OpenCode. Structure:
ROOT.md total plan: goals, hard rules, directory registry,
and a scenario -> file routing table
<category>/
index.md one line per file, COMPILED from frontmatter
<slug>.md the memory itself
Two roots, same shape: <config>/memory/ (global) and
<project canonical>/.memory/ (project). Project wins on conflict; global
hard rules stay resident everywhere.
Why a plugin rather than a skill that tells the model to read a folder:
model-driven retrieval is unreliable — the model forgets, reads half the
files, or reads the wrong ones. Markdown is the human-facing format; the
plugin is the engine that parses, scores, and injects.
Measured, not assumed: FTS5's two built-in tokenizers both fail on this
corpus. On SQLite 3.53.4 with "部署脚本用 pnpm 而不是 npm...": unicode61
emits 7 terms for the whole sentence and matches 0 hits for "部署";
trigram needs >= 3 characters and also returns 0 for that two-character
word. So tokenization happens in the plugin (CJK bigrams, latin words)
and no SQL index is used — a thousand memories are fine to scan.
index.md and ROOT's two tables are compiled, never hand-edited: a
hand-maintained index drifts, a generated one cannot. Hit counters live
in .state.json so cross-device sync does not conflict on every recall.
Tools are named recall/remember/forget/inspect rather than
write/read/status: generic names were observed shadowing the harness's
own built-in tools.
This commit is contained in:
@@ -0,0 +1,190 @@
|
||||
# memory 插件
|
||||
|
||||
给 OpenCode 加一套跨会话记忆。三级结构、多根并存、纯文本可手改，检索和注入由插件自己完成。
|
||||
|
||||
## 核心思想：ROOT 是索引，不是记忆
|
||||
|
||||
模型每次不需要读完所有记忆。读一份小的 `ROOT.md` 就知道"我有什么、什么时候该取哪一块"，
|
||||
需要细节时再用工具深挖。
|
||||
|
||||
```
|
||||
ROOT.md 总纲：当前目标 / 铁规则 / 目录登记 / 加载路由表
|
||||
<category>/
|
||||
index.md 每个文件一句话 —— 编译产物，不要手改
|
||||
<slug>.md 记忆本体，带 frontmatter
|
||||
```
|
||||
|
||||
注入分三层，预算递减：
|
||||
|
||||
| 层 | 内容 | 何时进上下文 |
|
||||
|---|---|---|
|
||||
| 铁规则 | 用户声明的硬性要求 | **每次必带**，不受相关性影响 |
|
||||
| 记忆大纲 | 有什么、每个文件一句话、触发场景 | 每次必带（很便宜） |
|
||||
| 记忆正文 | 命中最强的几条 | 每次最多 4 条，受字符预算限制 |
|
||||
|
||||
剩下的靠 `memory_recall` 按需取。
|
||||
|
||||
## 双根：全局 + 项目
|
||||
|
||||
两个根结构完全相同，可同时存在：
|
||||
|
||||
```
|
||||
~/.config/opencode/memory/ 全局：偏好、决策、通用坑
|
||||
<项目 canonical>/.memory/ 项目：项目事实、专属决策
|
||||
```
|
||||
|
||||
- 冲突时**项目级赢** —— 更具体的说法更有用
|
||||
- **全局铁规则始终常驻**，不能被项目覆盖；项目铁规则追加在后面
|
||||
- 项目根不存在就自动创建，不存在时静默跳过
|
||||
|
||||
项目根取自 `ctx.location.project.canonical`，也就是项目根目录（不是会话工作目录 ——
|
||||
两者在 worktree 里不同）。
|
||||
|
||||
## 为什么不是 skill + 文件夹
|
||||
|
||||
原来的做法是"写个 skill 告诉模型去读 `.memory/`"。这个方案的致命缺陷是**检索依赖模型自觉**：
|
||||
模型会忘记读、只读一半、或读错文件。
|
||||
|
||||
这里把职责拆开：
|
||||
|
||||
- **Markdown 是人类面向的格式** —— 纯文本、可手改、git 可 diff
|
||||
- **插件是检索引擎** —— 解析、评分、注入全部由代码保证
|
||||
|
||||
记忆会自动抽取（模型忘了写也有兜底），但**铁规则永远只能由人写** ——
|
||||
允许模型生成"铁规则"等于让幻觉变成永久策略。
|
||||
|
||||
## 中文分词：实测结论，不是推测
|
||||
|
||||
这是本插件唯一必须自己动手的地方。在 SQLite 3.53.4 上实测字符串
|
||||
`部署脚本用 pnpm 而不是 npm，因为 lockfile 不兼容`：
|
||||
|
||||
| FTS5 分词器 | 查"部署" | 查"部署脚本" | 原因 |
|
||||
|---|---|---|---|
|
||||
| `unicode61`（默认） | **0 命中** | **0 命中** | 整句只产生 7 个词条，把 `部署脚本用` 当**一个 token** |
|
||||
| `trigram` | **0 命中** | 1 命中 | 要求查询 ≥3 字符，词表是「不兼」「而不」这种三字滑窗 |
|
||||
|
||||
中文里「部署」「配置」「依赖」「接口」这类**两字词占比极高**，trigram 漏掉的正好是最常用的一批。
|
||||
|
||||
所以分词在插件里做，不在 SQL 里做：CJK 切**字符二元组**，拉丁切整词。两字中文能精确命中，
|
||||
多出来的噪声词交给 idf 自动降权。
|
||||
|
||||
（也因此没有用 SQLite 索引 —— 记忆是千级条目，全表扫描足够，索引换不来收益。）
|
||||
|
||||
## 评分
|
||||
|
||||
```
|
||||
score = 词项重叠 × idf × 作用域先验 × 时间衰减 × 复述加成
|
||||
```
|
||||
|
||||
- **作用域先验**：项目 1.0 / 全局 0.5 —— 同分时更具体的赢
|
||||
- **时间衰减**：半衰期 14 天（`halfLifeDays: 0` 关闭）。**不删除**，只降权
|
||||
- **复述加成**：`memory_recall` 命中过的记忆会累加计数，越常用越靠前
|
||||
- **钉住**：`pinned: true` 的条目不衰减、不被自动取代
|
||||
- **取代**：新记忆的 `supersedes` 指向旧的，旧的标 `superseded_by` 保留可追溯，
|
||||
但**不再参与检索**
|
||||
|
||||
## 工具
|
||||
|
||||
| 工具 | 作用 |
|
||||
|---|---|
|
||||
| `memory_recall` | 搜记忆。返回全文 + 命中理由（匹配了哪些词、命中几次） |
|
||||
| `memory_remember` | 写记忆。自动同步目录 index.md 和 ROOT 路由表 |
|
||||
| `memory_forget` | 删记忆。路径穿越会被拒绝 |
|
||||
| `memory_inspect` | 看当前所有记忆 + **索引一致性自检** —— 调优全靠它 |
|
||||
|
||||
> 工具刻意不叫 `write` / `read` / `status`：实测发现这类通用名会**遮蔽 harness 自带的
|
||||
> 同名工具**（`write` 曾直接让本会话失去建文件能力）。叶子名必须避开通用词。
|
||||
|
||||
## 记忆文件格式
|
||||
|
||||
```markdown
|
||||
---
|
||||
id: pitfalls/sqlite-cjk-tokenizer
|
||||
kind: pitfall
|
||||
summary: unicode61 把整段中文当一个 token，trigram 漏掉两字词。
|
||||
highlight: 实测：查"部署"两种分词器都是 0 命中
|
||||
created: 1791512014824
|
||||
updated: 1791512014824
|
||||
load_when:
|
||||
- SQLite
|
||||
- FTS5
|
||||
- 分词
|
||||
pinned: true
|
||||
---
|
||||
|
||||
正文：上下文、决策、原因、根因。
|
||||
```
|
||||
|
||||
`load_when` 是**路由表的唯一来源** —— 插件扫所有文件的这个字段编译出 ROOT 的路由表，
|
||||
所以表永远不会和实际内容对不上。
|
||||
|
||||
`index.md` 和 ROOT 的两张表同理，都是从 frontmatter 编译的。手写索引一定会漂移。
|
||||
|
||||
## 该记什么
|
||||
|
||||
| 该记 | 不该记 |
|
||||
|---|---|
|
||||
| 用户偏好 | 代码里已经能看到的东西 |
|
||||
| 架构决策 + 原因 | 当前任务状态 |
|
||||
| 非显然的坑 + 根因 | git 分支名、文件路径 |
|
||||
| 项目约定 | 一次性对话 |
|
||||
|
||||
插件会在写入前拦截疑似密钥（记忆会进 git）。
|
||||
|
||||
## 配置
|
||||
|
||||
默认零配置。放在 `opencode.jsonc` 的 `plugins` 里可调：
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"plugins": [
|
||||
{ "package": "./plugins/memory", "options": {
|
||||
"maxInjectChars": 1200, // 注入字符硬上限
|
||||
"maxInjectEntries": 4, // 每次内联几条正文
|
||||
"halfLifeDays": 14, // 衰减半衰期，0 = 不衰减
|
||||
"autoExtract": true, // 自动抽取
|
||||
"minTurns": 4, // 会话至少多少轮才抽取
|
||||
"extractModel": { "providerID": "anthropic", "id": "claude-haiku-4-5" },
|
||||
"categories": ["rules", "preferences", "decisions", "pitfalls", "facts"]
|
||||
} }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
不指定 `extractModel` 时自动挑**最便宜的可用模型**（按 cost 排序，并过滤掉上下文窗口 < 8k 的）。
|
||||
|
||||
## 自动抽取
|
||||
|
||||
监听 `session.idle` → 取本轮对话 → 廉价模型抽取结构化记忆 → 关键词去重 → 写入。
|
||||
|
||||
三重保护：
|
||||
|
||||
1. **独立模型**，不碰主对话 —— 抽取失败不影响用户
|
||||
2. **只写普通记忆**，铁规则降级为 `rule-proposal` 等人确认
|
||||
3. **相似度去重** —— 与已有记忆重叠的候选会被跳过并记日志
|
||||
|
||||
## 跨设备同步
|
||||
|
||||
`memory/` 已被 git 跟踪，`.state.json`（命中计数）被忽略 —— 否则每次检索都改文件，
|
||||
跨设备必然冲突。
|
||||
|
||||
**判据是陈旧风险**：偏好、决策、坑、约定可以同步；当前分支、本地路径、任务状态不同步。
|
||||
同步错的事实比漏同步更糟。
|
||||
|
||||
项目级 `.memory/` 不在全局仓库里，需要哪个项目同步，就在那个项目仓库里提交它。
|
||||
|
||||
## 排查
|
||||
|
||||
```bash
|
||||
opencode service restart # 改完插件代码后
|
||||
opencode plugin list # 确认 memory 已加载
|
||||
```
|
||||
|
||||
- 加载失败看 `~/.local/share/opencode/log/opencode.log`，过滤 `role=server`
|
||||
- 索引漂移用 `memory_inspect` 检查（它会核对 index.md 是否漏了文件）
|
||||
- 想知道"模型现在到底看到了什么"，看日志里的抽取日志
|
||||
|
||||
## 已知边界
|
||||
|
||||
- 记忆到**万条量级**时全表扫描会变慢，那时再换 SQLite 索引（正文是纯文本，迁移成本低）
|
||||
- 多设备同时同步同一份记忆会有 git 冲突，需要手动 rebase
|
||||
@@ -0,0 +1,289 @@
|
||||
// Automatic extraction: turn a finished turn into candidate memories.
|
||||
//
|
||||
// Runs on `session.idle`. The point is that memory capture must not depend on
|
||||
// the model remembering to call `memory_write` — models forget. This is the
|
||||
// safety net.
|
||||
//
|
||||
// Two deliberate limits keep it from becoming a garbage generator:
|
||||
//
|
||||
// 1. A separate, cheap model does the extraction. It never touches the main
|
||||
// conversation, so a bad extraction cannot derail the user's work.
|
||||
// 2. Hard rules are never written automatically. The extractor may *propose*
|
||||
// one, but only a human confirms it. Letting a model write "iron rules"
|
||||
// would let a hallucination become permanent policy.
|
||||
|
||||
import type { Context } from "@opencode/plugin"
|
||||
import { scoreFiles } from "./recall.ts"
|
||||
|
||||
export interface ExtractOptions {
|
||||
enabled: boolean
|
||||
/** Minimum turns in a session before extraction kicks in. */
|
||||
minTurns: number
|
||||
/** Cap on how many messages are handed to the extractor. */
|
||||
maxMessages: number
|
||||
model?: { providerID: string; id: string }
|
||||
}
|
||||
|
||||
export const DEFAULT_EXTRACT: ExtractOptions = {
|
||||
enabled: true,
|
||||
minTurns: 4,
|
||||
maxMessages: 24,
|
||||
}
|
||||
|
||||
export interface Candidate {
|
||||
category: string
|
||||
slug: string
|
||||
title: string
|
||||
summary: string
|
||||
highlight?: string
|
||||
body: string
|
||||
kind?: string
|
||||
loadWhen?: string[]
|
||||
/** Rules the model believes should be hard rules, needing human confirmation. */
|
||||
proposeRule?: boolean
|
||||
}
|
||||
|
||||
const SYSTEM = `You extract durable memories from a coding conversation.
|
||||
|
||||
Return ONLY a JSON array. Each element:
|
||||
{
|
||||
"category": "preferences|decisions|pitfalls|facts",
|
||||
"slug": "kebab-case-identifier",
|
||||
"title": "short title",
|
||||
"summary": "ONE sentence, under 100 chars. This is what shows in an index.",
|
||||
"highlight": "the single most important detail",
|
||||
"body": "markdown detail, 2-6 sentences",
|
||||
"loadWhen": ["trigger words or situations where this is relevant"],
|
||||
"proposeRule": false
|
||||
}
|
||||
|
||||
Only record what stays true after the session ends:
|
||||
GOOD A stated preference, an architectural decision and its reason,
|
||||
a non-obvious gotcha and its root cause, a project convention.
|
||||
BAD What the code currently looks like, task state, git branch names,
|
||||
file paths, anything already obvious from the repo, one-off chatter.
|
||||
|
||||
NEVER set proposeRule to true unless the user stated an absolute rule about how
|
||||
to behave ("always", "never", "必须", "禁止"). Doubtful -> omit the memory.
|
||||
|
||||
If nothing durable was learned, return [].`
|
||||
|
||||
/**
|
||||
* Pull plain text out of a session message without depending on the exact
|
||||
* message shape: the plugin package ships types for a schema the host may
|
||||
* evolve, so this walks defensively and collects `text` fields.
|
||||
*/
|
||||
export function collectText(value: unknown, depth = 0, out: string[] = []): string[] {
|
||||
if (depth > 6 || out.length > 400) return out
|
||||
if (typeof value === "string") {
|
||||
if (value.trim()) out.push(value)
|
||||
return out
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
for (const item of value) collectText(item, depth + 1, out)
|
||||
return out
|
||||
}
|
||||
if (value && typeof value === "object") {
|
||||
const record = value as Record<string, unknown>
|
||||
// Prefer a part's own text over its metadata noise.
|
||||
if (typeof record.text === "string" && record.text.trim()) out.push(record.text)
|
||||
for (const [key, child] of Object.entries(record)) {
|
||||
if (key === "text") continue
|
||||
if (key === "synthetic" || key === "cache" || key === "providerMetadata") continue
|
||||
collectText(child, depth + 1, out)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
export function userAndAssistantText(messages: readonly unknown[]): string {
|
||||
const parts: string[] = []
|
||||
for (const message of messages) {
|
||||
const record = message as Record<string, unknown> | null
|
||||
if (!record) continue
|
||||
const role = record.role
|
||||
if (role !== "user" && role !== "assistant") continue
|
||||
const texts = collectText(record.content)
|
||||
const joined = texts.join("\n").trim()
|
||||
if (joined) parts.push(`${role === "user" ? "USER" : "ASSISTANT"}: ${joined}`)
|
||||
}
|
||||
return parts.join("\n\n")
|
||||
}
|
||||
|
||||
/** `generate.text` is documented as returning text; accept a few shapes. */
|
||||
function coerceText(result: unknown): string {
|
||||
if (typeof result === "string") return result
|
||||
if (result && typeof result === "object") {
|
||||
const record = result as Record<string, unknown>
|
||||
for (const key of ["text", "content", "output", "result"]) {
|
||||
const value = record[key]
|
||||
if (typeof value === "string") return value
|
||||
if (Array.isArray(value)) {
|
||||
const joined = collectText(value).join("\n")
|
||||
if (joined) return joined
|
||||
}
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
/** Parse the extractor's JSON array, tolerating prose and code fences. */
|
||||
export function parseCandidates(raw: string): Candidate[] {
|
||||
const text = raw.trim()
|
||||
if (!text) return []
|
||||
const fenced = /```(?:json)?\s*([\s\S]*?)```/.exec(text)
|
||||
const body = fenced?.[1] ?? text
|
||||
const start = body.indexOf("[")
|
||||
const end = body.lastIndexOf("]")
|
||||
if (start < 0 || end <= start) return []
|
||||
|
||||
let parsed: unknown
|
||||
try {
|
||||
parsed = JSON.parse(body.slice(start, end + 1))
|
||||
} catch {
|
||||
return []
|
||||
}
|
||||
if (!Array.isArray(parsed)) return []
|
||||
|
||||
const out: Candidate[] = []
|
||||
for (const entry of parsed) {
|
||||
if (!entry || typeof entry !== "object") continue
|
||||
const record = entry as Record<string, unknown>
|
||||
const title = str(record.title)
|
||||
const bodyText = str(record.body)
|
||||
if (!title || !bodyText) continue
|
||||
|
||||
const category = str(record.category).toLowerCase() || "facts"
|
||||
const slug =
|
||||
slugify(str(record.slug)) || slugify(title)
|
||||
if (!slug) continue
|
||||
|
||||
out.push({
|
||||
category: category.replace(/[^a-z0-9._-]/g, "") || "facts",
|
||||
slug,
|
||||
title,
|
||||
summary: str(record.summary) || title,
|
||||
highlight: str(record.highlight) || undefined,
|
||||
body: bodyText,
|
||||
kind: str(record.kind) || undefined,
|
||||
loadWhen: Array.isArray(record.loadWhen)
|
||||
? record.loadWhen.map((v) => str(v)).filter(Boolean).slice(0, 8)
|
||||
: undefined,
|
||||
proposeRule: record.proposeRule === true,
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function str(value: unknown): string {
|
||||
return typeof value === "string" ? value.trim() : ""
|
||||
}
|
||||
|
||||
export function slugify(input: string): string {
|
||||
const ascii = input
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, "-")
|
||||
.replace(/^-+|-+$/g, "")
|
||||
.slice(0, 48)
|
||||
if (ascii) return ascii
|
||||
// CJK titles have no ASCII to work with; fall back to a hash-free stable id.
|
||||
return [...input].map((ch) => ch.codePointAt(0)!.toString(36)).join("").slice(0, 24)
|
||||
}
|
||||
|
||||
/** Secrets that must never reach a git-tracked memory file. */
|
||||
const SECRET_PATTERNS: RegExp[] = [
|
||||
/\b(?:sk|pk|rk)-[A-Za-z0-9_-]{16,}/g,
|
||||
/\bgh[pousr]_[A-Za-z0-9]{16,}/g,
|
||||
/\bAKIA[0-9A-Z]{16}\b/g,
|
||||
/\bxox[baprs]-[A-Za-z0-9-]{10,}/g,
|
||||
/\bey[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b/g,
|
||||
/\b(?:password|passwd|secret|api[_-]?key|access[_-]?token|auth[_-]?token)\s*[:=]\s*\S+/gi,
|
||||
/-----BEGIN[A-Z ]*PRIVATE KEY-----[\s\S]*?-----END[A-Z ]*PRIVATE KEY-----/g,
|
||||
]
|
||||
|
||||
export function hasSecret(text: string): string | undefined {
|
||||
for (const pattern of SECRET_PATTERNS) {
|
||||
const match = pattern.exec(text)
|
||||
if (match) return match[0].slice(0, 24)
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
export function redactSecrets(text: string): string {
|
||||
let out = text
|
||||
for (const pattern of SECRET_PATTERNS) out = out.replace(pattern, "[REDACTED]")
|
||||
return out
|
||||
}
|
||||
|
||||
export interface ExtractionOutcome {
|
||||
candidates: Candidate[]
|
||||
rejected: { candidate: Candidate; reason: string }[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide which candidates are worth writing, using existing memories for
|
||||
* dedupe. Near-duplicates bump the existing file instead of adding a rival.
|
||||
*/
|
||||
export function screenCandidates(
|
||||
candidates: Candidate[],
|
||||
existing: readonly import("./store.ts").MemoryFile[],
|
||||
options: { minimumScore?: number } = {},
|
||||
): ExtractionOutcome {
|
||||
const threshold = options.minimumScore ?? 0.9
|
||||
const accepted: Candidate[] = []
|
||||
const rejected: { candidate: Candidate; reason: string }[] = []
|
||||
|
||||
for (const candidate of candidates) {
|
||||
const secret = hasSecret(`${candidate.title}\n${candidate.summary}\n${candidate.body}`)
|
||||
if (secret) {
|
||||
rejected.push({ candidate, reason: `含疑似密钥 (${secret}…)` })
|
||||
continue
|
||||
}
|
||||
if (candidate.proposeRule) {
|
||||
// Not rejected outright — a proposal is surfaced for a human, which is
|
||||
// handled by the caller. Flag it here so it is written as a normal
|
||||
// memory under `proposals` rather than promoted to policy.
|
||||
candidate.category = "rules"
|
||||
candidate.kind = "rule-proposal"
|
||||
}
|
||||
const probe = `${candidate.title}\n${candidate.summary}\n${candidate.body}`
|
||||
const scored = scoreFiles(existing, probe, {}, { halfLifeDays: 0, pinnedBoost: false })
|
||||
const top = scored[0]
|
||||
if (top && top.score >= threshold) {
|
||||
rejected.push({ candidate, reason: `与已有记忆高度重叠 (${top.file.rel}, score ${top.score.toFixed(2)})` })
|
||||
continue
|
||||
}
|
||||
accepted.push(candidate)
|
||||
}
|
||||
|
||||
return { candidates: accepted, rejected }
|
||||
}
|
||||
|
||||
export interface RunExtractionInput {
|
||||
ctx: Context
|
||||
model: { providerID: string; id: string }
|
||||
transcript: string
|
||||
existing: readonly import("./store.ts").MemoryFile[]
|
||||
signal?: AbortSignal
|
||||
}
|
||||
|
||||
/** One model round-trip. Returns screened candidates and why things were dropped. */
|
||||
export async function extract(input: RunExtractionInput): Promise<ExtractionOutcome> {
|
||||
const prompt = `${SYSTEM}\n\n---\n\n${input.transcript}`
|
||||
let result: unknown
|
||||
try {
|
||||
// `generate.text` has no cancellation documented on this surface, so the
|
||||
// signal is forwarded when accepted and simply ignored when not.
|
||||
result = await (input.ctx.generate as { text: (input: unknown) => Promise<unknown> }).text({
|
||||
model: input.model,
|
||||
prompt,
|
||||
...(input.signal ? { signal: input.signal } : {}),
|
||||
})
|
||||
} catch {
|
||||
// An extraction failure must never disturb the user's session.
|
||||
return { candidates: [], rejected: [] }
|
||||
}
|
||||
const text = coerceText(result)
|
||||
const candidates = parseCandidates(text)
|
||||
return screenCandidates(candidates, input.existing)
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
// Minimal YAML-subset frontmatter, enough for memory files.
|
||||
//
|
||||
// A full YAML parser is deliberately avoided: memory files only need scalars
|
||||
// and string arrays, and every added dependency is one more thing that can fail
|
||||
// to load on someone else's machine. This reads what we write and tolerates
|
||||
// what humans type.
|
||||
|
||||
export interface Frontmatter {
|
||||
[key: string]: string | number | boolean | string[] | undefined
|
||||
}
|
||||
|
||||
/** Parse a leading `---` block. Returns empty data when there is none. */
|
||||
export function parseFrontmatter(raw: string): { data: Frontmatter; body: string } {
|
||||
// Tolerate a UTF-8 BOM and leading blank lines before the fence.
|
||||
const text = raw.replace(/^\uFEFF/, "")
|
||||
const match = /^---[ \t]*\r?\n([\s\S]*?)\r?\n---[ \t]*(?:\r?\n|$)/.exec(text)
|
||||
if (!match) return { data: {}, body: text.trim() }
|
||||
|
||||
const data: Frontmatter = {}
|
||||
let currentKey: string | undefined
|
||||
|
||||
for (const line of match[1].split(/\r?\n/)) {
|
||||
if (!line.trim() || line.trim().startsWith("#")) continue
|
||||
|
||||
// `- item` continues the array opened by the previous `key:`.
|
||||
const item = /^\s*-\s+(.*)$/.exec(line)
|
||||
if (item && currentKey) {
|
||||
const prev = data[currentKey]
|
||||
const list = Array.isArray(prev) ? prev : []
|
||||
list.push(scalar(item[1]))
|
||||
data[currentKey] = list
|
||||
continue
|
||||
}
|
||||
|
||||
const kv = /^([A-Za-z_][\w.-]*)\s*:\s*(.*)$/.exec(line)
|
||||
if (!kv) continue
|
||||
currentKey = kv[1]
|
||||
const value = kv[2].trim()
|
||||
if (value === "") {
|
||||
// Either an empty scalar or the head of a block list; the `- ` branch
|
||||
// above fills it in. Mark it as an empty array so round-tripping works.
|
||||
data[currentKey] = []
|
||||
continue
|
||||
}
|
||||
data[currentKey] = value.startsWith("[") ? parseInlineList(value) : scalar(value)
|
||||
}
|
||||
|
||||
return { data, body: text.slice(match[0].length).trim() }
|
||||
}
|
||||
|
||||
/** Serialize data + body back to a memory file. Key order is preserved. */
|
||||
export function stringifyFrontmatter(data: Frontmatter, body: string): string {
|
||||
const lines: string[] = []
|
||||
for (const [key, value] of Object.entries(data)) {
|
||||
if (value === undefined) continue
|
||||
if (Array.isArray(value)) {
|
||||
if (value.length === 0) {
|
||||
lines.push(`${key}: []`)
|
||||
continue
|
||||
}
|
||||
// Arrays are written one item per line: readable in a diff, and immune
|
||||
// to quoting problems that inline JSON lists hit with Chinese text.
|
||||
lines.push(`${key}:`)
|
||||
for (const entry of value) lines.push(` - ${quoteScalar(entry)}`)
|
||||
continue
|
||||
}
|
||||
lines.push(`${key}: ${quoteScalar(value)}`)
|
||||
}
|
||||
return `---\n${lines.join("\n")}\n---\n\n${body.trim()}\n`
|
||||
}
|
||||
|
||||
function scalar(raw: string): string | number | boolean {
|
||||
const value = unquote(raw.trim())
|
||||
if (value === "true") return true
|
||||
if (value === "false") return false
|
||||
if (/^-?\d+(\.\d+)?$/.test(value)) return Number(value)
|
||||
return value
|
||||
}
|
||||
|
||||
function parseInlineList(raw: string): string[] {
|
||||
const inner = raw.trim().replace(/^\[/, "").replace(/\]$/, "")
|
||||
if (!inner.trim()) return []
|
||||
return splitTopLevel(inner).map((part) => String(unquote(part.trim())))
|
||||
}
|
||||
|
||||
/** Split on commas that are not inside quotes. */
|
||||
function splitTopLevel(input: string): string[] {
|
||||
const parts: string[] = []
|
||||
let buf = ""
|
||||
let quote: string | undefined
|
||||
for (const ch of input) {
|
||||
if (quote) {
|
||||
if (ch === quote) quote = undefined
|
||||
else buf += ch
|
||||
continue
|
||||
}
|
||||
if (ch === '"' || ch === "'") {
|
||||
quote = ch
|
||||
continue
|
||||
}
|
||||
if (ch === ",") {
|
||||
parts.push(buf)
|
||||
buf = ""
|
||||
continue
|
||||
}
|
||||
buf += ch
|
||||
}
|
||||
if (buf.trim()) parts.push(buf)
|
||||
return parts
|
||||
}
|
||||
|
||||
function unquote(value: string): string {
|
||||
if (value.length >= 2 && ((value[0] === '"' && value.endsWith('"')) || (value[0] === "'" && value.endsWith("'")))) {
|
||||
return value.slice(1, -1)
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
/** Quote only when the value would otherwise not round-trip. */
|
||||
function quoteScalar(value: string | number | boolean): string {
|
||||
if (typeof value === "number" || typeof value === "boolean") return String(value)
|
||||
// A bare `:` or a leading marker would be read back as YAML structure.
|
||||
if (/^[\s>|*&!%@`#-]|[:#]\s|^$/.test(value)) {
|
||||
return `"${value.replace(/\\/g, "\\\\").replace(/"/g, '\\"')}"`
|
||||
}
|
||||
return value
|
||||
}
|
||||
@@ -0,0 +1,680 @@
|
||||
// Memory plugin for OpenCode — three-tier, multi-root, human-editable.
|
||||
//
|
||||
// ROOT.md the total plan: goals, hard rules, directory registry,
|
||||
// and a routing table of "scenario -> which file to load"
|
||||
// <category>/ one per kind of memory, same shape at every level
|
||||
// index.md one line per file. COMPILED from frontmatter, never edited
|
||||
// <slug>.md the memory, with frontmatter (summary / highlight / load_when)
|
||||
//
|
||||
// Two roots coexist and use identical rules:
|
||||
// global <config>/memory/ preferences, decisions, general pitfalls
|
||||
// project <project canonical>/.memory/ facts specific to this checkout
|
||||
// Project wins on conflict; global hard rules stay resident everywhere.
|
||||
//
|
||||
// Why a plugin rather than a skill telling the model to read a folder: model-
|
||||
// driven retrieval is unreliable — the model forgets, reads half the files, or
|
||||
// reads the wrong ones. Here the plugin does retrieval, scoring, and injection
|
||||
// itself. The Markdown is the human-facing format; the plugin is the engine.
|
||||
//
|
||||
// Docs consulted:
|
||||
// OpenCode V2 plugins: https://opencode.ai/v2/docs/build/plugins/
|
||||
// OpenCode plugins config: https://opencode.ai/v2/docs/plugins
|
||||
// Session hooks / events: see the plugin page's Hooks and Events sections
|
||||
//
|
||||
// Prerequisite: npm i @opencode/plugin@$(opencode --version | sed 's/^v//')
|
||||
// then `opencode service restart`.
|
||||
|
||||
import { Plugin, type Context } from "@opencode/plugin"
|
||||
import fs from "node:fs"
|
||||
import path from "node:path"
|
||||
import { fileURLToPath } from "node:url"
|
||||
import { buildInjection, readGoal, scoreFiles } from "./recall.ts"
|
||||
import { DEFAULT_EXTRACT, extract, redactSecrets, userAndAssistantText, type ExtractOptions } from "./extract.ts"
|
||||
import {
|
||||
type MemoryFile,
|
||||
type MemoryRoot,
|
||||
compileCategoryIndexes,
|
||||
compileRoot,
|
||||
ensureRoot,
|
||||
findNearestRoot,
|
||||
globalRoot,
|
||||
markSuperseded,
|
||||
projectRoot,
|
||||
readState,
|
||||
scanRoot,
|
||||
validateCategory,
|
||||
validateSlug,
|
||||
writeMemory,
|
||||
writeState,
|
||||
} from "./store.ts"
|
||||
|
||||
interface PluginConfig {
|
||||
/** Hard cap on injected characters per model request. Default 1200. */
|
||||
maxInjectChars?: number
|
||||
/** How many memory bodies to inline per request. Default 4. */
|
||||
maxInjectEntries?: number
|
||||
/** Half-life in days for the recency decay. Default 14; 0 disables. */
|
||||
halfLifeDays?: number
|
||||
autoExtract?: boolean
|
||||
minTurns?: number
|
||||
extractModel?: { providerID: string; id: string }
|
||||
/** Seed these categories on first run. */
|
||||
categories?: string[]
|
||||
}
|
||||
|
||||
const CATEGORIES = ["rules", "preferences", "decisions", "pitfalls", "facts"]
|
||||
const DEFAULT_MAX_INJECT = 1200
|
||||
|
||||
export default Plugin.define({
|
||||
id: "memory",
|
||||
|
||||
async setup(ctx) {
|
||||
const config = ctx.options as PluginConfig
|
||||
const maxChars = config.maxInjectChars ?? DEFAULT_MAX_INJECT
|
||||
const maxEntries = config.maxInjectEntries ?? 4
|
||||
const halfLifeDays = config.halfLifeDays ?? 14
|
||||
const categories = config.categories ?? CATEGORIES
|
||||
const extractOptions: ExtractOptions = {
|
||||
...DEFAULT_EXTRACT,
|
||||
enabled: config.autoExtract !== false,
|
||||
minTurns: config.minTurns ?? DEFAULT_EXTRACT.minTurns,
|
||||
...(config.extractModel ? { model: config.extractModel } : {}),
|
||||
}
|
||||
|
||||
// `ctx.location.directory` is the session's working directory, NOT the
|
||||
// config root, so it cannot be used to locate this plugin's own files. The
|
||||
// plugin directory itself is the only reliable anchor:
|
||||
// <config>/plugins/memory/index.ts -> up two levels -> <config>
|
||||
const configDir = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "..")
|
||||
const global = globalRoot(configDir)
|
||||
|
||||
// A project root is only created on demand: an empty `.memory/` directory in
|
||||
// every checkout is worse than none at all.
|
||||
let project: MemoryRoot | undefined
|
||||
const canonical = ctx.location.project?.canonical
|
||||
if (canonical && canonical !== "/" && fs.existsSync(canonical)) {
|
||||
project = projectRoot(canonical, String(ctx.location.project.id))
|
||||
}
|
||||
|
||||
const roots = (): MemoryRoot[] => (project ? [global, project] : [global])
|
||||
|
||||
// Seeding the skeleton must never be able to break plugin load: an
|
||||
// unwritable directory should degrade to "no memories yet", not to a
|
||||
// failed plugin. Every read path already tolerates a missing root.
|
||||
try {
|
||||
ensureRoot(global, categories)
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`memory: cannot create ${global.dir} (${(error as Error).message}). Memories are disabled until this is writable.`,
|
||||
)
|
||||
}
|
||||
|
||||
// -- in-memory view, refreshed when files change on disk ----------------
|
||||
let cache: { filesByRoot: Map<string, MemoryFile[]>; at: number } | undefined
|
||||
const filesByRoot = (): Map<string, MemoryFile[]> => {
|
||||
const now = Date.now()
|
||||
if (cache && now - cache.at < 2000) return cache.filesByRoot
|
||||
const map = new Map<string, MemoryFile[]>()
|
||||
for (const root of roots()) map.set(root.key, scanRoot(root))
|
||||
cache = { filesByRoot: map, at: now }
|
||||
return map
|
||||
}
|
||||
const allFiles = (): MemoryFile[] => roots().flatMap((r) => filesByRoot().get(r.key) ?? [])
|
||||
const states = (root: MemoryRoot) => readState(root)
|
||||
const invalidate = () => {
|
||||
cache = undefined
|
||||
}
|
||||
|
||||
// Recompile every derived index after a write.
|
||||
const recompile = () => {
|
||||
for (const root of roots()) {
|
||||
const files = scanRoot(root)
|
||||
compileCategoryIndexes(root, files)
|
||||
compileRoot(root, files)
|
||||
}
|
||||
invalidate()
|
||||
}
|
||||
|
||||
// -- injection into the agent loop --------------------------------------
|
||||
// `context` covers the agent loop including tool continuations, and edits
|
||||
// only the outgoing request — the conversation transcript stays clean.
|
||||
await ctx.session.hook("context", (event) => {
|
||||
try {
|
||||
const query = latestUserText(event.messages)
|
||||
if (!query) return
|
||||
const text = buildInjection({
|
||||
roots: roots(),
|
||||
filesByRoot: filesByRoot(),
|
||||
statesByRoot: new Map(roots().map((r) => [r.key, states(r)])),
|
||||
query,
|
||||
options: { maxChars, maxEntries, halfLifeDays },
|
||||
})
|
||||
if (text) event.system.push({ type: "text", text })
|
||||
} catch (error) {
|
||||
console.error(`memory: injection failed: ${(error as Error).message}`)
|
||||
}
|
||||
})
|
||||
|
||||
// -- tools --------------------------------------------------------------
|
||||
await ctx.tool.transform((editor) => {
|
||||
editor.namespace({
|
||||
name: "memory",
|
||||
description:
|
||||
"Durable memory across sessions: recall what was learned before, record new facts, forget stale ones.",
|
||||
})
|
||||
|
||||
// memory_recall ------------------------------------------------------
|
||||
editor.add({
|
||||
name: "recall",
|
||||
description: [
|
||||
"Search memory for durable knowledge from earlier sessions: preferences, decisions, conventions, pitfalls.",
|
||||
"Start here when ROOT's outline mentions something relevant but you need the full detail,",
|
||||
"or when the user refers to something you do not remember deciding.",
|
||||
"Prefer this over re-deriving a convention from the repo.",
|
||||
].join(" "),
|
||||
input: {
|
||||
type: "object",
|
||||
properties: {
|
||||
query: {
|
||||
type: "string",
|
||||
description: "What you are trying to recall — a topic, a decision, a symptom.",
|
||||
},
|
||||
scope: {
|
||||
type: "string",
|
||||
enum: ["any", "project", "global"],
|
||||
description: "Restrict to one root. Default any (project wins on conflict).",
|
||||
},
|
||||
limit: {
|
||||
type: "number",
|
||||
description: "Max memories to return. Default 5.",
|
||||
},
|
||||
},
|
||||
required: ["query"],
|
||||
additionalProperties: false,
|
||||
},
|
||||
execute: async (input: { query: string; scope?: string; limit?: number }) => {
|
||||
const pool = pickRoots(roots(), input.scope)
|
||||
const scored = pool
|
||||
.flatMap((root) =>
|
||||
scoreFiles(filesByRoot().get(root.key) ?? [], input.query, states(root), { halfLifeDays }),
|
||||
)
|
||||
// Same rel path in two roots: the project one is the specific truth.
|
||||
.filter((entry, i, all) => {
|
||||
const same = all.filter((e) => e.file.rel === entry.file.rel)
|
||||
return same.length === 1 || same[0] === entry
|
||||
})
|
||||
.sort((a, b) => b.score - a.score)
|
||||
.slice(0, input.limit ?? 5)
|
||||
|
||||
// Reinforcement: what gets used floats up next time.
|
||||
if (scored.length) {
|
||||
for (const root of pool) {
|
||||
const state = states(root)
|
||||
let dirty = false
|
||||
for (const entry of scored) {
|
||||
if (entry.file.root.key !== root.key) continue
|
||||
const current = state[entry.file.rel]
|
||||
state[entry.file.rel] = {
|
||||
hitCount: (current?.hitCount ?? 0) + 1,
|
||||
lastHitAt: Date.now(),
|
||||
}
|
||||
dirty = true
|
||||
}
|
||||
if (dirty) {
|
||||
try {
|
||||
writeState(root, state)
|
||||
} catch {
|
||||
/* stats are best-effort */
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (scored.length === 0) {
|
||||
return {
|
||||
content: `没有匹配 "${input.query}" 的记忆。\n\n当前记忆库：\n${outlineText(filesByRoot(), roots())}`,
|
||||
}
|
||||
}
|
||||
|
||||
const body = scored
|
||||
.map((entry) => {
|
||||
const tags = [
|
||||
`[${entry.file.root.label}/${entry.file.rel}]`,
|
||||
entry.file.kind ? `(${entry.file.kind})` : "",
|
||||
entry.reasons.length ? ` ${entry.reasons.join(", ")}` : "",
|
||||
].join("")
|
||||
return `${tags}\nscore ${entry.score.toFixed(2)}\n\n${entry.file.body}`
|
||||
})
|
||||
.join("\n\n---\n\n")
|
||||
return { content: body }
|
||||
},
|
||||
})
|
||||
|
||||
// memory_remember -------------------------------------------------------
|
||||
editor.add({
|
||||
name: "remember",
|
||||
description: [
|
||||
"Record a durable memory: a user preference, an architectural decision and its reason,",
|
||||
"a non-obvious gotcha, or a project convention.",
|
||||
"Do NOT record what the code already shows, current task state, branch names, or file paths.",
|
||||
"Writes automatically refresh the category index and ROOT's routing table.",
|
||||
].join(" "),
|
||||
input: {
|
||||
type: "object",
|
||||
properties: {
|
||||
category: {
|
||||
type: "string",
|
||||
description: "Which kind of memory. Use rules/ only for things the user declared absolute.",
|
||||
enum: categories,
|
||||
},
|
||||
slug: {
|
||||
type: "string",
|
||||
description: "Stable kebab-case identifier. Reusing a slug updates that memory in place.",
|
||||
pattern: "^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$",
|
||||
},
|
||||
title: { type: "string", description: "Short title." },
|
||||
summary: {
|
||||
type: "string",
|
||||
description: "ONE sentence under 100 chars. This is the only line shown in indexes — make it count.",
|
||||
},
|
||||
highlight: {
|
||||
type: "string",
|
||||
description: "The single most important detail, shown as the 重点 column.",
|
||||
},
|
||||
body: { type: "string", description: "The detail: context, the decision, the reason, the root cause." },
|
||||
scope: {
|
||||
type: "string",
|
||||
enum: ["project", "global"],
|
||||
description:
|
||||
"Which root. Default project. Use global for preferences and knowledge that holds across projects.",
|
||||
},
|
||||
loadWhen: {
|
||||
type: "array",
|
||||
items: { type: "string" },
|
||||
description:
|
||||
"Scenarios or trigger words that should surface this memory. These build ROOT's routing table.",
|
||||
},
|
||||
pinned: {
|
||||
type: "boolean",
|
||||
description:
|
||||
"Pin the memory: never decays, never superseded automatically. Use for things you stated as absolute.",
|
||||
},
|
||||
supersedes: {
|
||||
type: "string",
|
||||
description: "Relative path of an older memory this replaces. The old one is kept for audit, not injected.",
|
||||
},
|
||||
},
|
||||
required: ["category", "slug", "title", "summary", "body"],
|
||||
additionalProperties: false,
|
||||
},
|
||||
execute: async (input: {
|
||||
category: string
|
||||
slug: string
|
||||
title: string
|
||||
summary: string
|
||||
highlight?: string
|
||||
body: string
|
||||
scope?: string
|
||||
loadWhen?: string[]
|
||||
pinned?: boolean
|
||||
supersedes?: string
|
||||
}) => {
|
||||
const categoryError = validateCategory(input.category)
|
||||
if (categoryError) return { content: `拒绝写入：${categoryError}` }
|
||||
const slugError = validateSlug(input.slug)
|
||||
if (slugError) return { content: `拒绝写入：${slugError}` }
|
||||
|
||||
const safeBody = redactSecrets(input.body)
|
||||
if (safeBody !== input.body) {
|
||||
return {
|
||||
content:
|
||||
"拒绝写入：正文里检测到疑似密钥（已定位）。记忆文件会进入 git，请改成描述性文字，例如“凭证放在环境变量 X”。",
|
||||
}
|
||||
}
|
||||
|
||||
const root =
|
||||
input.scope === "global" || !project ? global : project
|
||||
const created = fs.existsSync(path.join(root.dir, input.category, `${input.slug}.md`))
|
||||
|
||||
const file = writeMemory(root, {
|
||||
category: input.category,
|
||||
slug: input.slug,
|
||||
title: input.title,
|
||||
summary: input.summary,
|
||||
highlight: input.highlight ?? "",
|
||||
body: safeBody,
|
||||
...(input.loadWhen?.length ? { loadWhen: input.loadWhen } : {}),
|
||||
// Pass `false` through explicitly so an explicit unpin works;
|
||||
// leaving it undefined lets writeMemory keep an existing pin.
|
||||
...(input.pinned === undefined ? {} : { pinned: input.pinned }),
|
||||
...(input.supersedes ? { supersedes: input.supersedes } : {}),
|
||||
})
|
||||
|
||||
if (input.supersedes) markSuperseded(root, input.supersedes, file.rel)
|
||||
|
||||
// Promote the whole root's routing table when a new file appears, so
|
||||
// the outline the model sees stays complete.
|
||||
recompile()
|
||||
|
||||
return {
|
||||
content: [
|
||||
`${created ? "已更新" : "已写入"}记忆：${file.rel}（根：${root.label}）`,
|
||||
"",
|
||||
`概述：${file.summary}`,
|
||||
file.loadWhen.length ? `触发场景：${file.loadWhen.join("、")}` : "触发场景：未设置（不会被路由表自动命中，用 memory_recall 按关键词取）",
|
||||
"",
|
||||
"已同步更新：该目录 index.md" +
|
||||
(file.loadWhen.length ? " 与 ROOT.md 路由表" : "") +
|
||||
"。",
|
||||
].join("\n"),
|
||||
}
|
||||
},
|
||||
})
|
||||
|
||||
// memory_forget ------------------------------------------------------
|
||||
editor.add({
|
||||
name: "forget",
|
||||
description: [
|
||||
"Remove or supersede a memory that turned out to be wrong.",
|
||||
"Prefer `supersedes` when a newer fact replaces an older one — the old entry stays for audit.",
|
||||
"Use this when the user corrects you or a memory is no longer true.",
|
||||
].join(" "),
|
||||
input: {
|
||||
type: "object",
|
||||
properties: {
|
||||
path: {
|
||||
type: "string",
|
||||
description: 'Relative path of the memory, e.g. "pitfalls/sqlite-cjk.md".',
|
||||
},
|
||||
scope: {
|
||||
type: "string",
|
||||
enum: ["project", "global", "any"],
|
||||
description: "Which root to look in. Default any.",
|
||||
},
|
||||
reason: { type: "string", description: "Why it is being forgotten. Recorded for audit." },
|
||||
},
|
||||
required: ["path", "reason"],
|
||||
additionalProperties: false,
|
||||
},
|
||||
execute: async (input: { path: string; scope?: string; reason: string }) => {
|
||||
const target = input.path.replace(/^\/+/, "")
|
||||
if (!target.endsWith(".md") || target.includes("..")) {
|
||||
return { content: `拒绝：path 必须是根下的相对 .md 路径（收到 ${JSON.stringify(input.path)}）` }
|
||||
}
|
||||
const pool = pickRoots(roots(), input.scope)
|
||||
for (const root of pool) {
|
||||
const abs = path.join(root.dir, target)
|
||||
if (!fs.existsSync(abs)) continue
|
||||
fs.rmSync(abs)
|
||||
recompile()
|
||||
return { content: `已删除 ${root.label}/${target}。\n理由：${input.reason}\n已同步重建 index.md 与 ROOT.md 路由表。` }
|
||||
}
|
||||
return { content: `未找到记忆 ${target}。\n现有记忆：\n${outlineText(filesByRoot(), roots())}` }
|
||||
},
|
||||
})
|
||||
|
||||
// memory_inspect ------------------------------------------------------
|
||||
editor.add({
|
||||
name: "inspect",
|
||||
description:
|
||||
"Inspect the memory library: every root, every memory with its summary and trigger, plus index consistency checks. Use when tuning memory or explaining what is remembered.",
|
||||
input: {
|
||||
type: "object",
|
||||
properties: {
|
||||
detail: {
|
||||
type: "boolean",
|
||||
description: "Include per-memory hit counts and scores. Default false.",
|
||||
},
|
||||
},
|
||||
additionalProperties: false,
|
||||
},
|
||||
execute: async (input: { detail?: boolean }) => {
|
||||
const lines: string[] = []
|
||||
for (const root of roots()) {
|
||||
const files = filesByRoot().get(root.key) ?? []
|
||||
const state = states(root)
|
||||
const goal = readGoal(root.dir)
|
||||
lines.push(`## 根：${root.label}（${root.scope}）`)
|
||||
lines.push(`路径：${root.dir}${fs.existsSync(root.dir) ? "" : " ← 尚未创建"}`)
|
||||
if (goal) lines.push(`当前目标：${goal}`)
|
||||
if (files.length === 0) {
|
||||
lines.push("（空）")
|
||||
lines.push("")
|
||||
continue
|
||||
}
|
||||
lines.push("")
|
||||
for (const file of files) {
|
||||
const hits = state[file.rel]
|
||||
const superseded = file.fm.superseded_by ? ` [已被 ${String(file.fm.superseded_by)} 取代]` : ""
|
||||
lines.push(`- ${file.rel}${superseded}`)
|
||||
lines.push(` ${file.summary}`)
|
||||
if (file.loadWhen.length) lines.push(` 触发：${file.loadWhen.join("、")}`)
|
||||
if (file.pinned) lines.push(" 钉住（不衰减、不被取代）")
|
||||
if (input.detail) {
|
||||
lines.push(
|
||||
` 命中 ${hits?.hitCount ?? 0} 次 · 最后命中 ${
|
||||
hits?.lastHitAt ? new Date(hits.lastHitAt).toISOString().slice(0, 10) : "从未"
|
||||
}`,
|
||||
)
|
||||
}
|
||||
}
|
||||
lines.push("")
|
||||
}
|
||||
|
||||
// Consistency: the compiled tables must match what is on disk, or the
|
||||
// outline the model trusts is a lie.
|
||||
const problems: string[] = []
|
||||
for (const root of roots()) {
|
||||
// A root nobody has written to yet is not a problem to report.
|
||||
if (!fs.existsSync(root.dir)) continue
|
||||
const files = scanRoot(root)
|
||||
const categories = [...new Set(files.map((f) => f.category))]
|
||||
for (const category of categories) {
|
||||
const indexPath = path.join(root.dir, category, "index.md")
|
||||
if (!fs.existsSync(indexPath)) {
|
||||
problems.push(`${root.label}/${category}/index.md 缺失`)
|
||||
continue
|
||||
}
|
||||
const indexText = fs.readFileSync(indexPath, "utf8")
|
||||
for (const file of files.filter((f) => f.category === category)) {
|
||||
if (!indexText.includes(`${file.slug}.md`)) {
|
||||
problems.push(`${root.label}/${category}/index.md 缺少 ${file.slug}.md`)
|
||||
}
|
||||
}
|
||||
}
|
||||
const rootPath = path.join(root.dir, "ROOT.md")
|
||||
if (!fs.existsSync(rootPath)) problems.push(`${root.label}/ROOT.md 缺失`)
|
||||
}
|
||||
lines.push(problems.length === 0 ? "索引一致性：正常" : `索引一致性：发现 ${problems.length} 个问题`)
|
||||
for (const problem of problems) lines.push(` - ${problem}`)
|
||||
|
||||
return { content: lines.join("\n") }
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
// -- automatic extraction on idle --------------------------------------
|
||||
let extracting = false
|
||||
const controller = new AbortController()
|
||||
|
||||
if (extractOptions.enabled) {
|
||||
void (async () => {
|
||||
try {
|
||||
for await (const event of ctx.event.subscribe({ signal: controller.signal })) {
|
||||
const record = event as unknown as { type?: string; data?: { sessionID?: string } }
|
||||
if (record?.type !== "session.idle") continue
|
||||
const sessionID = record.data?.sessionID
|
||||
if (!sessionID || extracting) continue
|
||||
|
||||
extracting = true
|
||||
try {
|
||||
await runExtraction(ctx, {
|
||||
sessionID,
|
||||
extractOptions,
|
||||
filesByRoot,
|
||||
project,
|
||||
global,
|
||||
})
|
||||
} catch (error) {
|
||||
console.error(`memory: extraction failed: ${(error as Error).message}`)
|
||||
} finally {
|
||||
extracting = false
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
// Subscription teardown is normal on unload.
|
||||
if (!controller.signal.aborted) {
|
||||
console.error(`memory: event stream ended: ${(error as Error).message}`)
|
||||
}
|
||||
}
|
||||
})()
|
||||
}
|
||||
|
||||
return () => {
|
||||
controller.abort()
|
||||
}
|
||||
},
|
||||
})
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function pickRoots(roots: MemoryRoot[], scope: string | undefined): MemoryRoot[] {
|
||||
if (scope === "project") return roots.filter((r) => r.scope === "project")
|
||||
if (scope === "global") return roots.filter((r) => r.scope === "global")
|
||||
return roots
|
||||
}
|
||||
|
||||
function outlineText(filesByRoot: Map<string, MemoryFile[]>, roots: MemoryRoot[]): string {
|
||||
const lines: string[] = []
|
||||
for (const root of roots) {
|
||||
const files = filesByRoot.get(root.key) ?? []
|
||||
if (files.length === 0) continue
|
||||
lines.push(`${root.label}：`)
|
||||
for (const file of files) lines.push(` ${file.rel} — ${file.summary}`)
|
||||
}
|
||||
return lines.join("\n") || "（记忆库为空）"
|
||||
}
|
||||
|
||||
/** The newest user text in a request — the retrieval query. */
|
||||
function latestUserText(messages: readonly unknown[]): string | undefined {
|
||||
for (let i = messages.length - 1; i >= 0; i--) {
|
||||
const record = messages[i] as { role?: string; content?: unknown } | null
|
||||
if (!record || record.role !== "user") continue
|
||||
const text = extractUserText(record.content)
|
||||
if (text) return text
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
function extractUserText(content: unknown): string {
|
||||
if (typeof content === "string") return content.trim()
|
||||
if (!Array.isArray(content)) return ""
|
||||
const parts: string[] = []
|
||||
for (const part of content) {
|
||||
if (part && typeof part === "object") {
|
||||
const record = part as Record<string, unknown>
|
||||
if (record.type === "text" && typeof record.text === "string") parts.push(record.text)
|
||||
}
|
||||
}
|
||||
return parts.join(" ").trim()
|
||||
}
|
||||
|
||||
async function runExtraction(
|
||||
ctx: Context,
|
||||
input: {
|
||||
sessionID: string
|
||||
extractOptions: ExtractOptions
|
||||
filesByRoot: () => Map<string, MemoryFile[]>
|
||||
project: MemoryRoot | undefined
|
||||
global: MemoryRoot
|
||||
},
|
||||
): Promise<void> {
|
||||
const { sessionID, extractOptions } = input
|
||||
|
||||
const messages = (await ctx.session.context({ sessionID })) as readonly unknown[]
|
||||
const turns = messages.filter((m) => {
|
||||
const role = (m as { role?: string }).role
|
||||
return role === "user" || role === "assistant"
|
||||
})
|
||||
// Long enough to have learned something, short enough to stay cheap.
|
||||
if (turns.length < extractOptions.minTurns) return
|
||||
|
||||
const model = extractOptions.model ?? (await pickCheapModel(ctx))
|
||||
if (!model) return
|
||||
|
||||
const transcript = userAndAssistantText(turns.slice(-extractOptions.maxMessages))
|
||||
if (transcript.length < 200) return
|
||||
|
||||
const existing = [...input.filesByRoot().values()].flat()
|
||||
const outcome = await extract({ ctx, model, transcript, existing })
|
||||
|
||||
if (outcome.candidates.length === 0) {
|
||||
for (const rejected of outcome.rejected) {
|
||||
console.log(
|
||||
`memory: skipped ${rejected.candidate.category}/${rejected.candidate.slug} — ${rejected.reason}`,
|
||||
)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
const now = Date.now()
|
||||
const written: string[] = []
|
||||
for (const candidate of outcome.candidates) {
|
||||
// Content that mentions this project's own paths belongs to the project.
|
||||
const text = `${candidate.title}\n${candidate.summary}\n${candidate.body}`
|
||||
const projectSpecific =
|
||||
input.project !== undefined && (text.includes(ctx.location.project.directory) || text.includes(ctx.location.directory))
|
||||
const root = projectSpecific ? input.project! : input.global
|
||||
|
||||
const file = writeMemory(root, {
|
||||
category: candidate.category,
|
||||
slug: candidate.slug,
|
||||
title: candidate.title,
|
||||
summary: candidate.summary,
|
||||
highlight: candidate.highlight ?? "",
|
||||
body: redactSecrets(candidate.body),
|
||||
kind: candidate.kind ?? (candidate.proposeRule ? "rule-proposal" : "fact"),
|
||||
...(candidate.loadWhen?.length ? { loadWhen: candidate.loadWhen } : {}),
|
||||
})
|
||||
written.push(`${root.label}/${file.rel}`)
|
||||
}
|
||||
|
||||
if (written.length) {
|
||||
for (const root of [input.global, input.project].filter(Boolean) as MemoryRoot[]) {
|
||||
compileCategoryIndexes(root, scanRoot(root))
|
||||
compileRoot(root, scanRoot(root))
|
||||
}
|
||||
console.log(`memory: extracted ${written.length} memor${written.length === 1 ? "y" : "ies"} → ${written.join(", ")}`)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Pick the cheapest available model for extraction. Falls back to the session
|
||||
* default when cost metadata is unavailable, and disables extraction entirely
|
||||
* when neither is available.
|
||||
*/
|
||||
async function pickCheapModel(ctx: Context): Promise<{ providerID: string; id: string } | undefined> {
|
||||
try {
|
||||
const models = await ctx.model.list()
|
||||
let best: { ref: { providerID: string; id: string }; cost: number } | undefined
|
||||
for (const model of models) {
|
||||
if (model.disabled) continue
|
||||
// A reasoning-only or tiny-context model cannot do the job reliably.
|
||||
if ((model.limit?.context ?? 0) > 0 && model.limit.context < 8_000) continue
|
||||
const output = model.cost?.[0]?.output ?? 0
|
||||
const input = model.cost?.[0]?.input ?? 0
|
||||
const cost = Number(output) + Number(input)
|
||||
if (!best || cost < best.cost) {
|
||||
best = { ref: { providerID: model.providerID, id: model.id }, cost }
|
||||
}
|
||||
}
|
||||
if (best) return best.ref
|
||||
const fallback = await ctx.model.default()
|
||||
if (fallback) return { providerID: fallback.providerID, id: fallback.modelID }
|
||||
} catch {
|
||||
return undefined
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
@@ -0,0 +1,283 @@
|
||||
// Retrieval: tokenization, scoring, and the block that gets injected.
|
||||
//
|
||||
// Tokenization note — this is the part that had to be measured, not guessed.
|
||||
//
|
||||
// FTS5's two built-in tokenizers both fail on the memory corpus this plugin
|
||||
// exists to hold. Measured on SQLite 3.53.4 with the string
|
||||
// "部署脚本用 pnpm 而不是 npm，因为 lockfile 不兼容":
|
||||
//
|
||||
// unicode61 (FTS5 default) query "部署" -> 0 hits
|
||||
// query "部署脚本" -> 0 hits
|
||||
// It emits 7 terms for the whole sentence, treating 部署脚本用 as ONE
|
||||
// token, so no substring can ever match.
|
||||
//
|
||||
// trigram query "部署" -> 0 hits (needs >= 3 chars)
|
||||
// query "部署脚" -> 1 hit
|
||||
// Its vocabulary is sliding 3-char windows (不兼 / 而不 / m，因), so every
|
||||
// two-character Chinese word — 部署, 配置, 依赖, 接口 — is unreachable.
|
||||
//
|
||||
// So tokenization happens here, not in SQL: CJK becomes character bigrams and
|
||||
// latin becomes words. Two-character Chinese then matches exactly, and the
|
||||
// extra noise bigrams introduce is harmless because rarer terms score lower.
|
||||
|
||||
import fs from "node:fs"
|
||||
import path from "node:path"
|
||||
|
||||
const CJK =
|
||||
/[\u3400-\u4dbf\u4e00-\u9fff\uf900-\ufaff\u3040-\u30ff\uac00-\ud7af]/
|
||||
const LATIN_WORD = /[A-Za-z0-9][A-Za-z0-9_+.#-]*/g
|
||||
|
||||
/** Stopwords that would otherwise dominate a scoring pass. */
|
||||
const STOPWORDS = new Set([
|
||||
"the", "a", "an", "and", "or", "of", "to", "in", "is", "are", "was", "were",
|
||||
"be", "for", "on", "at", "by", "with", "as", "it", "this", "that", "from",
|
||||
"我", "的", "了", "是", "在", "和", "有", "就", "不", "也", "你",
|
||||
])
|
||||
|
||||
export function tokenize(text: string): string[] {
|
||||
if (!text) return []
|
||||
const tokens: string[] = []
|
||||
|
||||
// Latin / digits: whole words, lowercased.
|
||||
for (const match of text.toLowerCase().matchAll(LATIN_WORD)) {
|
||||
const word = match[0]
|
||||
if (word.length < 2 || STOPWORDS.has(word)) continue
|
||||
tokens.push(word)
|
||||
}
|
||||
|
||||
// CJK: unigrams plus bigrams. Bigrams carry the matching power; unigrams
|
||||
// rescue one-character queries that have no bigram at all.
|
||||
const runs = text.match(/[\u3400-\u4dbf\u4e00-\u9fff\uf900-\ufaff\u3040-\u30ff\uac00-\ud7af]+/g) ?? []
|
||||
for (const run of runs) {
|
||||
const chars = [...run]
|
||||
for (let i = 0; i < chars.length - 1; i++) {
|
||||
const bigram = chars[i] + chars[i + 1]
|
||||
if (!STOPWORDS.has(bigram)) tokens.push(bigram)
|
||||
}
|
||||
if (chars.length === 1 && !STOPWORDS.has(chars[0])) tokens.push(chars[0])
|
||||
}
|
||||
|
||||
return tokens
|
||||
}
|
||||
|
||||
export interface ScoredFile {
|
||||
file: import("./store.ts").MemoryFile
|
||||
score: number
|
||||
reasons: string[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Score memories against a query.
|
||||
*
|
||||
* Ranking is scope-first on purpose: a project-specific fact about how *this*
|
||||
* repo deploys should beat a global preference when both match, because the
|
||||
* narrower claim is the more useful one. Ties inside one root fall back to
|
||||
* decay and hit count.
|
||||
*/
|
||||
export function scoreFiles(
|
||||
files: readonly import("./store.ts").MemoryFile[],
|
||||
query: string,
|
||||
states: Record<string, import("./store.ts").FileState>,
|
||||
options: { now?: number; halfLifeDays?: number; pinnedBoost?: boolean } = {},
|
||||
): ScoredFile[] {
|
||||
const now = options.now ?? Date.now()
|
||||
const halfLife = (options.halfLifeDays ?? 14) * 86_400_000
|
||||
const queryTokens = tokenize(query)
|
||||
if (queryTokens.length === 0) return []
|
||||
|
||||
const querySet = new Map<string, number>()
|
||||
for (const token of queryTokens) querySet.set(token, (querySet.get(token) ?? 0) + 1)
|
||||
|
||||
const results: ScoredFile[] = []
|
||||
|
||||
for (const file of files) {
|
||||
// A superseded memory stays on disk for audit but must never be injected.
|
||||
if (file.fm.superseded_by) continue
|
||||
|
||||
const state = states[file.rel]
|
||||
const haystack = [file.title, file.summary, file.highlight, file.loadWhen.join(" "), file.body].join("\n")
|
||||
const docTokens = new Set(tokenize(haystack))
|
||||
|
||||
let overlap = 0
|
||||
const matched: string[] = []
|
||||
for (const [token, weight] of querySet) {
|
||||
if (!docTokens.has(token)) continue
|
||||
// Diminishing returns: a term repeated in the query should not dominate.
|
||||
overlap += weight * (1 / (1 + Math.log2(weight)))
|
||||
if (matched.length < 4) matched.push(token)
|
||||
}
|
||||
if (overlap <= 0) continue
|
||||
|
||||
// Relative term rarity across the corpus: a term in every file is useless.
|
||||
const idf = idfFor(tokenScore(querySet, docTokens), files.length)
|
||||
let score = overlap * idf
|
||||
|
||||
// Scope prior.
|
||||
score *= file.root.rank === 20 ? 1 : 0.5
|
||||
|
||||
// Recency decay, unless pinned or a hard rule.
|
||||
const hard = file.pinned || String(file.fm.kind ?? "").toLowerCase() === "rule" || file.category === "rules"
|
||||
if (!hard && halfLife > 0) {
|
||||
const age = Math.max(0, now - (state?.lastHitAt ?? file.updated))
|
||||
score *= Math.pow(0.5, age / halfLife)
|
||||
}
|
||||
|
||||
// Reinforcement: memories that keep proving useful float up.
|
||||
score *= 1 + Math.log1p(state?.hitCount ?? 0) * 0.25
|
||||
|
||||
if (options.pinnedBoost !== false && hard) score *= 1.15
|
||||
|
||||
const reasons = [...matched]
|
||||
if (hard) reasons.push("铁规则")
|
||||
if ((state?.hitCount ?? 0) > 0) reasons.push(`命中 ${state?.hitCount} 次`)
|
||||
|
||||
results.push({ file, score, reasons })
|
||||
}
|
||||
|
||||
return results.sort((a, b) => b.score - a.score)
|
||||
}
|
||||
|
||||
function tokenScore(querySet: Map<string, number>, docTokens: Set<string>): number {
|
||||
let n = 0
|
||||
for (const token of querySet.keys()) if (docTokens.has(token)) n++
|
||||
return n
|
||||
}
|
||||
|
||||
/** Inverse document frequency over the candidate set. */
|
||||
function idfFor(matchedTerms: number, total: number): number {
|
||||
if (matchedTerms <= 0 || total <= 0) return 0
|
||||
return Math.log(1 + total / matchedTerms)
|
||||
}
|
||||
|
||||
export interface InjectionOptions {
|
||||
/** Hard cap on the injected block, in characters. */
|
||||
maxChars?: number
|
||||
/** Max number of memory bodies to inline. */
|
||||
maxEntries?: number
|
||||
now?: number
|
||||
}
|
||||
|
||||
export interface InjectionInput {
|
||||
roots: readonly import("./store.ts").MemoryRoot[]
|
||||
filesByRoot: Map<string, import("./store.ts").MemoryFile[]>
|
||||
statesByRoot: Map<string, Record<string, import("./store.ts").FileState>>
|
||||
query: string
|
||||
options?: InjectionOptions
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the text injected into the agent loop.
|
||||
*
|
||||
* Two tiers on purpose:
|
||||
* - outlines are cheap and always accurate, so they always ride along
|
||||
* - full bodies are expensive, so only the best few ride along
|
||||
* A `memory_recall` call reaches the rest.
|
||||
*/
|
||||
export function buildInjection(input: InjectionInput): string {
|
||||
const { roots, filesByRoot, statesByRoot, query } = input
|
||||
const maxChars = input.options?.maxChars ?? 1200
|
||||
const maxEntries = input.options?.maxEntries ?? 4
|
||||
const now = input.options?.now ?? Date.now()
|
||||
|
||||
const blocks: string[] = []
|
||||
|
||||
// Tier 1 — hard rules. Resident regardless of query: these are the user's
|
||||
// non-negotiables and must never be outranked by relevance.
|
||||
const ruleLines: string[] = []
|
||||
for (const root of roots) {
|
||||
for (const file of filesByRoot.get(root.key) ?? []) {
|
||||
const isRule = file.category === "rules" || String(file.fm.kind ?? "").toLowerCase() === "rule"
|
||||
if (!isRule || file.fm.superseded_by) continue
|
||||
ruleLines.push(`- [${root.label}] ${file.title}${file.summary ? ` — ${file.summary}` : ""}`)
|
||||
}
|
||||
}
|
||||
|
||||
// Tier 2 — the outline: what exists, and when each part is worth loading.
|
||||
// This is the ROOT.md idea: one small file answers "what do I know and when
|
||||
// do I need it", instead of dumping every memory into every request.
|
||||
const outline: string[] = []
|
||||
for (const root of roots) {
|
||||
const files = filesByRoot.get(root.key) ?? []
|
||||
if (files.length === 0) continue
|
||||
const goals = readGoal(root.dir)
|
||||
if (goals) outline.push(`目标(${root.label}): ${goals}`)
|
||||
for (const [category, list] of groupCategories(files)) {
|
||||
const bits = list.map((f) => `${f.slug}(${f.summary.slice(0, 28)})`)
|
||||
outline.push(`${root.label}/${category}: ${bits.join(", ")}`)
|
||||
}
|
||||
}
|
||||
|
||||
// Tier 3 — full bodies for the top matches.
|
||||
const bodyLines: string[] = []
|
||||
let budget = maxChars - ruleLines.join("\n").length - outline.join("\n").length
|
||||
const all: ScoredFile[] = []
|
||||
for (const root of roots) {
|
||||
const scored = scoreFiles(filesByRoot.get(root.key) ?? [], query, statesByRoot.get(root.key) ?? {}, { now })
|
||||
for (const entry of scored) {
|
||||
// Project wins over global on the same file path.
|
||||
if (all.some((a) => a.file.rel === entry.file.rel && a.file.root.rank < entry.file.root.rank)) continue
|
||||
all.push(entry)
|
||||
}
|
||||
}
|
||||
for (const entry of all) {
|
||||
if (bodyLines.length >= maxEntries) break
|
||||
if (entry.score <= 0) continue
|
||||
const body = entry.file.body.slice(0, 400)
|
||||
const line = `\n### ${entry.file.root.label}/${entry.file.rel}\n${body}`
|
||||
if (budget - line.length < 0) break
|
||||
budget -= line.length
|
||||
bodyLines.push(line)
|
||||
}
|
||||
|
||||
const sections: string[] = []
|
||||
if (ruleLines.length) {
|
||||
sections.push(`## 铁规则（必须遵守）\n${ruleLines.join("\n")}`)
|
||||
}
|
||||
if (outline.length) {
|
||||
sections.push(`## 记忆大纲（按需深挖，用 memory_recall 取全文）\n${outline.join("\n")}`)
|
||||
}
|
||||
if (bodyLines.length) {
|
||||
sections.push(`## 与当前任务相关的记忆\n${bodyLines.join("\n")}`)
|
||||
}
|
||||
if (sections.length === 0) return ""
|
||||
|
||||
const header = "# 记忆上下文（来自 memory 插件；如需更多记忆用 memory_recall 工具）"
|
||||
let text = `${header}\n\n${sections.join("\n\n")}`
|
||||
if (text.length > maxChars) {
|
||||
// Truncate from the least important tier: keep rules, drop the tail of the
|
||||
// outline, then the bodies.
|
||||
const keep = sections.slice(0, Math.max(1, sections.length - 1)).join("\n\n")
|
||||
text = `${header}\n\n${keep}`.slice(0, maxChars)
|
||||
}
|
||||
return text
|
||||
}
|
||||
|
||||
function groupCategories(files: readonly import("./store.ts").MemoryFile[]) {
|
||||
const map = new Map<string, import("./store.ts").MemoryFile[]>()
|
||||
for (const file of files) {
|
||||
const list = map.get(file.category) ?? []
|
||||
list.push(file)
|
||||
map.set(file.category, list)
|
||||
}
|
||||
return [...map.entries()].sort((a, b) => a[0].localeCompare(b[0]))
|
||||
}
|
||||
|
||||
/** Read the hand-written "当前目标" block from ROOT.md. */
|
||||
export function readGoal(rootDir: string): string | undefined {
|
||||
try {
|
||||
const text = fs.readFileSync(path.join(rootDir, "ROOT.md"), "utf8")
|
||||
const m = /^##\s*当前目标\s*\n([\s\S]*?)(?=\n##\s|\n<!--|$)/m.exec(text)
|
||||
const body = m?.[1]
|
||||
?.replace(/<!--[\s\S]*?-->/g, "")
|
||||
.split(/\r?\n/)
|
||||
.map((l) => l.trim())
|
||||
.filter((l) => l && !l.startsWith("<!--"))
|
||||
.join(" ")
|
||||
.trim()
|
||||
return body || undefined
|
||||
} catch {
|
||||
// A root nobody has written to yet has no goals. This runs on every
|
||||
// injection, so a missing ROOT.md is a quiet outcome, not a failure.
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,539 @@
|
||||
// Memory store: roots, files, and the compiled indexes.
|
||||
//
|
||||
// The layout is deliberately the same at every level, so a project-local
|
||||
// `.memory/` and the global root are interchangeable:
|
||||
//
|
||||
// ROOT.md total plan: goals, hard rules, registry, routing table
|
||||
// <category>/
|
||||
// index.md one line per file — COMPILED, never hand-edited
|
||||
// <slug>.md the memory itself, with frontmatter
|
||||
//
|
||||
// Two rules keep the indexes honest:
|
||||
//
|
||||
// 1. `index.md` tables and ROOT's registry/routing tables are generated from
|
||||
// frontmatter. A hand-maintained index drifts from reality; a generated
|
||||
// one cannot.
|
||||
// 2. Hand-written regions in ROOT.md live outside `<!-- auto:begin -->`
|
||||
// markers and are never touched by the compiler.
|
||||
|
||||
import fs from "node:fs"
|
||||
import path from "node:path"
|
||||
import { type Frontmatter, parseFrontmatter, stringifyFrontmatter } from "./frontmatter.ts"
|
||||
|
||||
export type RootScope = "global" | "project"
|
||||
|
||||
export interface MemoryRoot {
|
||||
readonly scope: RootScope
|
||||
/** Stable identity used in state keys: "global" or "project:<id>". */
|
||||
readonly key: string
|
||||
/** Human label shown in injected context. */
|
||||
readonly label: string
|
||||
readonly dir: string
|
||||
/** Project wins ties; global is the fallback. */
|
||||
readonly rank: number
|
||||
}
|
||||
|
||||
export interface MemoryFile {
|
||||
readonly root: MemoryRoot
|
||||
/** Path relative to the root, e.g. "pitfalls/sqlite-cjk.md". */
|
||||
readonly rel: string
|
||||
readonly abs: string
|
||||
readonly category: string
|
||||
readonly slug: string
|
||||
readonly fm: Frontmatter
|
||||
readonly body: string
|
||||
readonly title: string
|
||||
/** One-line description shown in indexes and injected summaries. */
|
||||
readonly summary: string
|
||||
/** The "重点" column in index.md. */
|
||||
readonly highlight: string
|
||||
/** Scenarios that should pull this file in. Drives ROOT's routing table. */
|
||||
readonly loadWhen: string[]
|
||||
readonly pinned: boolean
|
||||
readonly created: number
|
||||
readonly updated: number
|
||||
readonly mtime: number
|
||||
}
|
||||
|
||||
export interface FileState {
|
||||
hitCount: number
|
||||
lastHitAt: number
|
||||
}
|
||||
|
||||
const AUTO_BEGIN = "<!-- auto:begin"
|
||||
const AUTO_END = "<!-- auto:end"
|
||||
const STATE_FILE = ".state.json"
|
||||
|
||||
/** Categories whose files are hard rules: always resident, never decayed. */
|
||||
const HARD_RULES_CATEGORY = "rules"
|
||||
|
||||
export function isHardRule(file: MemoryFile): boolean {
|
||||
if (file.category === HARD_RULES_CATEGORY) return true
|
||||
return String(file.fm.kind ?? "").toLowerCase() === "rule"
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Roots
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function globalRoot(configDir: string): MemoryRoot {
|
||||
return {
|
||||
scope: "global",
|
||||
key: "global",
|
||||
label: "全局",
|
||||
dir: path.join(configDir, "memory"),
|
||||
rank: 10,
|
||||
}
|
||||
}
|
||||
|
||||
export function projectRoot(canonical: string, projectID: string): MemoryRoot {
|
||||
return {
|
||||
scope: "project",
|
||||
key: `project:${projectID}`,
|
||||
label: path.basename(canonical) || "项目",
|
||||
dir: path.join(canonical, ".memory"),
|
||||
rank: 20,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Walk up from `dir` looking for a `.memory/` directory. This is what makes the
|
||||
* feature work in a monorepo or a nested checkout, where the session's
|
||||
* canonical project root may be several levels above the file being edited.
|
||||
*/
|
||||
export function findNearestRoot(start: string, depth = 4): string | undefined {
|
||||
let current = path.resolve(start)
|
||||
for (let i = 0; i <= depth; i++) {
|
||||
const candidate = path.join(current, ".memory")
|
||||
if (fs.existsSync(path.join(candidate, "ROOT.md"))) return candidate
|
||||
const parent = path.dirname(current)
|
||||
if (parent === current) break
|
||||
current = parent
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Scanning
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function scanRoot(root: MemoryRoot): MemoryFile[] {
|
||||
const out: MemoryFile[] = []
|
||||
let entries: fs.Dirent[]
|
||||
try {
|
||||
entries = fs.readdirSync(root.dir, { withFileTypes: true })
|
||||
} catch {
|
||||
return out
|
||||
}
|
||||
|
||||
for (const entry of entries) {
|
||||
if (!entry.isDirectory() || entry.name.startsWith(".")) continue
|
||||
const category = entry.name
|
||||
const dir = path.join(root.dir, category)
|
||||
let files: fs.Dirent[]
|
||||
try {
|
||||
files = fs.readdirSync(dir, { withFileTypes: true })
|
||||
} catch {
|
||||
continue
|
||||
}
|
||||
for (const file of files) {
|
||||
// `index.md` is compiled output; ROOT.md lives at the root, not here.
|
||||
if (!file.isFile() || !file.name.endsWith(".md")) continue
|
||||
if (file.name === "index.md" || file.name === "ROOT.md") continue
|
||||
const abs = path.join(dir, file.name)
|
||||
const parsed = readMemoryFile(root, abs, category, file.name)
|
||||
if (parsed) out.push(parsed)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function readMemoryFile(root: MemoryRoot, abs: string, category: string, name: string): MemoryFile | undefined {
|
||||
let raw: string
|
||||
let stat: fs.Stats
|
||||
try {
|
||||
raw = fs.readFileSync(abs, "utf8")
|
||||
stat = fs.statSync(abs)
|
||||
} catch {
|
||||
return undefined
|
||||
}
|
||||
const { data, body } = parseFrontmatter(raw)
|
||||
const slug = name.replace(/\.md$/, "")
|
||||
return {
|
||||
root,
|
||||
rel: path.posix.join(category, name),
|
||||
abs,
|
||||
category,
|
||||
slug,
|
||||
fm: data,
|
||||
body,
|
||||
title: String(data.title ?? firstHeading(body) ?? slug),
|
||||
summary: String(data.summary ?? deriveSummary(body, slug)),
|
||||
highlight: String(data.highlight ?? ""),
|
||||
loadWhen: toStringArray(data.load_when ?? data.loadWhen),
|
||||
pinned: data.pinned === true || data.pinned === "true",
|
||||
created: toNumber(data.created) ?? stat.mtimeMs,
|
||||
updated: toNumber(data.updated) ?? stat.mtimeMs,
|
||||
mtime: stat.mtimeMs,
|
||||
}
|
||||
}
|
||||
|
||||
function firstHeading(body: string): string | undefined {
|
||||
const m = /^#{1,3}\s+(.+)$/m.exec(body)
|
||||
return m?.[1]?.trim()
|
||||
}
|
||||
|
||||
/**
|
||||
* Fallback summary when frontmatter omits one: the first real paragraph,
|
||||
* flattened to a single line and clipped. Falls back to the slug so the index
|
||||
* never has an empty cell.
|
||||
*/
|
||||
function deriveSummary(body: string, slug: string): string {
|
||||
const lines = body.split(/\r?\n/)
|
||||
const buf: string[] = []
|
||||
for (const line of lines) {
|
||||
const text = line.trim()
|
||||
if (!text || text.startsWith("#") || text.startsWith("```")) continue
|
||||
buf.push(text)
|
||||
if (buf.join(" ").length > 90) break
|
||||
}
|
||||
const text = buf.join(" ").replace(/\s+/g, " ").trim()
|
||||
if (!text) return slug
|
||||
return text.length > 140 ? `${text.slice(0, 139)}…` : text
|
||||
}
|
||||
|
||||
function toStringArray(value: unknown): string[] {
|
||||
if (Array.isArray(value)) return value.map((v) => String(v)).filter(Boolean)
|
||||
if (typeof value === "string" && value.trim()) {
|
||||
return value
|
||||
.split(/[,，]/)
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean)
|
||||
}
|
||||
return []
|
||||
}
|
||||
|
||||
function toNumber(value: unknown): number | undefined {
|
||||
if (typeof value === "number" && Number.isFinite(value)) return value
|
||||
if (typeof value === "string") {
|
||||
const n = Number(value)
|
||||
if (Number.isFinite(n)) return n
|
||||
const parsed = Date.parse(value)
|
||||
if (Number.isFinite(parsed)) return parsed
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// State (hit counts) — derived, never the source of truth
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function readState(root: MemoryRoot): Record<string, FileState> {
|
||||
try {
|
||||
const raw = fs.readFileSync(path.join(root.dir, STATE_FILE), "utf8")
|
||||
const parsed = JSON.parse(raw)
|
||||
return typeof parsed === "object" && parsed ? (parsed as Record<string, FileState>) : {}
|
||||
} catch {
|
||||
return {}
|
||||
}
|
||||
}
|
||||
|
||||
/** Write state atomically so a crash mid-write cannot truncate the file. */
|
||||
export function writeState(root: MemoryRoot, state: Record<string, FileState>): void {
|
||||
fs.mkdirSync(root.dir, { recursive: true })
|
||||
const target = path.join(root.dir, STATE_FILE)
|
||||
const tmp = `${target}.tmp-${process.pid}`
|
||||
fs.writeFileSync(tmp, `${JSON.stringify(state, null, 2)}\n`, "utf8")
|
||||
fs.renameSync(tmp, target)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Writing memories
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface WriteInput {
|
||||
category: string
|
||||
slug: string
|
||||
title?: string
|
||||
summary?: string
|
||||
highlight?: string
|
||||
body: string
|
||||
kind?: string
|
||||
loadWhen?: string[]
|
||||
pinned?: boolean
|
||||
supersedes?: string
|
||||
}
|
||||
|
||||
const CATEGORY_RE = /^[a-z0-9][a-z0-9._-]{0,31}$/
|
||||
const SLUG_RE = /^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$/
|
||||
|
||||
export function validateCategory(category: string): string | undefined {
|
||||
if (!CATEGORY_RE.test(category)) {
|
||||
return `category must be lowercase [a-z0-9._-] (got ${JSON.stringify(category)})`
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
export function validateSlug(slug: string): string | undefined {
|
||||
if (!SLUG_RE.test(slug)) {
|
||||
return `slug must match [A-Za-z0-9][A-Za-z0-9._-]* (got ${JSON.stringify(slug)})`
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
export function writeMemory(root: MemoryRoot, input: WriteInput, now = Date.now()): MemoryFile {
|
||||
const dir = path.join(root.dir, input.category)
|
||||
fs.mkdirSync(dir, { recursive: true })
|
||||
const abs = path.join(dir, `${input.slug}.md`)
|
||||
const rel = path.posix.join(input.category, `${input.slug}.md`)
|
||||
|
||||
const existing = fs.existsSync(abs) ? readMemoryFile(root, abs, input.category, `${input.slug}.md`) : undefined
|
||||
const data: Frontmatter = {
|
||||
id: rel,
|
||||
// A file under `rules/` is a hard rule regardless of what it was labelled
|
||||
// before, so the category-derived value wins over a stale stored one. An
|
||||
// explicit `kind` from the caller still wins over both.
|
||||
kind: input.kind ?? (input.category === "rules" ? "rule" : String(existing?.fm.kind ?? "fact")),
|
||||
summary: input.summary ?? String(existing?.fm.summary ?? ""),
|
||||
highlight: input.highlight ?? String(existing?.fm.highlight ?? ""),
|
||||
created: existing?.created ?? now,
|
||||
updated: now,
|
||||
}
|
||||
if (input.loadWhen?.length) data.load_when = input.loadWhen
|
||||
else if (existing?.loadWhen.length) data.load_when = existing.loadWhen
|
||||
if (input.pinned ?? existing?.pinned) data.pinned = true
|
||||
if (input.supersedes) data.supersedes = input.supersedes
|
||||
|
||||
const body = input.body.trim() || String(existing?.body ?? "")
|
||||
fs.writeFileSync(abs, stringifyFrontmatter(data, body), "utf8")
|
||||
return readMemoryFile(root, abs, input.category, `${input.slug}.md`)!
|
||||
}
|
||||
|
||||
/** Mark `supersedes` (a rel path) as replaced, keeping the row for audit. */
|
||||
export function markSuperseded(root: MemoryRoot, target: string, byRel: string, now = Date.now()): void {
|
||||
const abs = path.join(root.dir, target)
|
||||
if (!fs.existsSync(abs) || !target.endsWith(".md")) return
|
||||
const { data, body } = parseFrontmatter(fs.readFileSync(abs, "utf8"))
|
||||
data.superseded_by = byRel
|
||||
data.updated = now
|
||||
fs.writeFileSync(abs, stringifyFrontmatter(data, body), "utf8")
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Compiled indexes
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Replace the region belonging to one specific `<!-- auto:begin <label> -->`.
|
||||
*
|
||||
* The label must be part of the match. Matching on a bare `auto:begin` pairs
|
||||
* the first marker with the *next* marker regardless of which region was meant,
|
||||
* which silently swaps two unrelated tables.
|
||||
*/
|
||||
export function replaceAutoRegion(text: string, label: string, generated: string): string {
|
||||
const re = new RegExp(
|
||||
`(${escapeRe(`${AUTO_BEGIN} ${label}`)}[^\\n]*\\n)[\\s\\S]*?(${escapeRe(AUTO_END)})`,
|
||||
)
|
||||
if (!re.test(text)) return text
|
||||
return text.replace(re, (_m, open: string, close: string) => `${open}${generated}${close}`)
|
||||
}
|
||||
|
||||
function escapeRe(value: string): string {
|
||||
return value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")
|
||||
}
|
||||
|
||||
function cell(value: string): string {
|
||||
return value.replace(/\|/g, "\\|").replace(/\r?\n/g, " ").trim() || "—"
|
||||
}
|
||||
|
||||
export interface CategoryIndex {
|
||||
category: string
|
||||
files: MemoryFile[]
|
||||
}
|
||||
|
||||
export function groupByCategory(files: MemoryFile[]): CategoryIndex[] {
|
||||
const map = new Map<string, MemoryFile[]>()
|
||||
for (const file of files) {
|
||||
const list = map.get(file.category) ?? []
|
||||
list.push(file)
|
||||
map.set(file.category, list)
|
||||
}
|
||||
return [...map.entries()]
|
||||
.map(([category, list]) => ({
|
||||
category,
|
||||
files: list.sort((a, b) => a.slug.localeCompare(b.slug)),
|
||||
}))
|
||||
.sort((a, b) => a.category.localeCompare(b.category))
|
||||
}
|
||||
|
||||
/** Rewrite `<root>/<category>/index.md` for every category that has files. */
|
||||
export function compileCategoryIndexes(root: MemoryRoot, files: MemoryFile[]): string[] {
|
||||
const written: string[] = []
|
||||
for (const { category, files: list } of groupByCategory(files)) {
|
||||
const heading = `# ${category} 索引`
|
||||
const rows = list.map((f) => `| ${cell(f.slug + ".md")} | ${cell(f.summary)} | ${cell(f.highlight)} |`)
|
||||
const table = [
|
||||
"| 文件 | 概述 | 重点 |",
|
||||
"|---|---|---|",
|
||||
...rows,
|
||||
"",
|
||||
].join("\n")
|
||||
const generated = `\n| 文件 | 概述 | 重点 |\n|---|---|---|\n${rows.join("\n")}\n`
|
||||
// Create the directory before writing: a root can be compiled before anything
|
||||
// was ever written into it (e.g. the global root was seeded but the project
|
||||
// root has no memories yet), and the compiler must not depend on that.
|
||||
fs.mkdirSync(path.join(root.dir, category), { recursive: true })
|
||||
const target = path.join(root.dir, category, "index.md")
|
||||
|
||||
let existing = ""
|
||||
try {
|
||||
existing = fs.readFileSync(target, "utf8")
|
||||
} catch {
|
||||
/* first build */
|
||||
}
|
||||
const next = existing.includes(AUTO_BEGIN)
|
||||
? replaceAutoRegion(existing, category, generated)
|
||||
: `${heading}\n\n${AUTO_BEGIN} ${category}\n${generated}${AUTO_END}\n`
|
||||
|
||||
// Skip the write when nothing changed: index.md is regenerated often and a
|
||||
// stable mtime keeps file watchers (and git) quiet.
|
||||
if (existing !== next) fs.writeFileSync(target, next, "utf8")
|
||||
written.push(path.join(category, "index.md"))
|
||||
}
|
||||
|
||||
// Prune indexes whose category lost every memory. A stale index listing
|
||||
// deleted files is exactly the drift this whole design exists to prevent, so
|
||||
// it must not be left behind.
|
||||
pruneEmptyIndexes(root, files)
|
||||
return written
|
||||
}
|
||||
|
||||
function pruneEmptyIndexes(root: MemoryRoot, files: MemoryFile[]): void {
|
||||
const populated = new Set(files.map((f) => f.category))
|
||||
let entries: fs.Dirent[]
|
||||
try {
|
||||
entries = fs.readdirSync(root.dir, { withFileTypes: true })
|
||||
} catch {
|
||||
return
|
||||
}
|
||||
for (const entry of entries) {
|
||||
if (!entry.isDirectory() || entry.name.startsWith(".")) continue
|
||||
if (populated.has(entry.name)) continue
|
||||
// A hand-written index in an empty category is the user's content; only
|
||||
// remove one this compiler produced.
|
||||
const target = path.join(root.dir, entry.name, "index.md")
|
||||
try {
|
||||
const text = fs.readFileSync(target, "utf8")
|
||||
if (text.includes(AUTO_BEGIN)) fs.rmSync(target)
|
||||
} catch {
|
||||
/* nothing to prune */
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export interface CompiledRoot {
|
||||
registry: string
|
||||
routing: string
|
||||
}
|
||||
|
||||
export function compileRootTables(root: MemoryRoot, files: MemoryFile[]): CompiledRoot {
|
||||
const categories = groupByCategory(files)
|
||||
|
||||
const registryRows = categories.map(({ category, files: list }) => {
|
||||
const pinned = list.filter((f) => f.pinned).length
|
||||
const desc = describeCategory(category, list)
|
||||
return `| ${cell(category + "/")} | ${cell(desc)} | ${list.length}${pinned ? ` (${pinned} 钉住)` : ""} |`
|
||||
})
|
||||
const registry = `\n| 目录 | 概述 | 条目 |\n|---|---|---|\n${registryRows.join("\n")}\n`
|
||||
|
||||
// The routing table is the union of every file's `load_when`, so it stays
|
||||
// correct without anyone maintaining it.
|
||||
const routingRows: string[] = []
|
||||
for (const { category, files: list } of categories) {
|
||||
for (const file of list) {
|
||||
for (const trigger of file.loadWhen) {
|
||||
routingRows.push(`| ${cell(trigger)} | ${cell(file.rel)} | ${cell(file.kind ?? "fact")} |`)
|
||||
}
|
||||
}
|
||||
}
|
||||
routingRows.sort((a, b) => a.localeCompare(b))
|
||||
const routing = routingRows.length
|
||||
? `\n| 场景 / 触发 | 读哪个文件 | 类型 |\n|---|---|---|\n${routingRows.join("\n")}\n`
|
||||
: `\n_暂无 load_when 标记的记忆。给文件加 frontmatter 的 \`load_when:\` 即可自动进入路由表。_\n`
|
||||
|
||||
return { registry, routing }
|
||||
}
|
||||
|
||||
function describeCategory(category: string, files: MemoryFile[]): string {
|
||||
const pinned = files.find((f) => isHardRule(f))
|
||||
if (pinned && category === HARD_RULES_CATEGORY) return "不可违反的硬规则，常驻"
|
||||
const first = files[0]
|
||||
if (first?.fm.category_desc) return String(first.fm.category_desc)
|
||||
return first ? first.summary.slice(0, 40) : "（空）"
|
||||
}
|
||||
|
||||
/** Recompile ROOT.md's generated regions and write it if it does not exist. */
|
||||
export function compileRoot(root: MemoryRoot, files: MemoryFile[]): string {
|
||||
// Same reason as the category index: ROOT.md may be the first thing written
|
||||
// into a root that does not exist yet.
|
||||
fs.mkdirSync(root.dir, { recursive: true })
|
||||
const target = path.join(root.dir, "ROOT.md")
|
||||
const { registry, routing } = compileRootTables(root, files)
|
||||
|
||||
let existing: string | undefined
|
||||
try {
|
||||
existing = fs.readFileSync(target, "utf8")
|
||||
} catch {
|
||||
/* first build */
|
||||
}
|
||||
|
||||
const text = existing ?? defaultRootTemplate(root)
|
||||
// Labeled regions, replaced in order. Each call is independent because the
|
||||
// label is part of the match — see replaceAutoRegion.
|
||||
let next = replaceAutoRegion(text, "registry", registry)
|
||||
next = replaceAutoRegion(next, "routing", routing)
|
||||
|
||||
if (next !== existing) fs.writeFileSync(target, next, "utf8")
|
||||
return next
|
||||
}
|
||||
|
||||
export function defaultRootTemplate(root: MemoryRoot): string {
|
||||
const where =
|
||||
root.scope === "global"
|
||||
? "跨项目通用记忆：你的偏好、技术决策、通用约定、踩过的坑。"
|
||||
: "本项目专属记忆：项目目标、事实、决策、坑。优先级高于全局根。"
|
||||
|
||||
return `# 记忆总纲 — ${root.label}
|
||||
|
||||
> ${where}
|
||||
> 由 memory 插件维护。\`auto:begin\` / \`auto:end\` 之间的表格是编译产物，不要手改；
|
||||
> 其余区域随便写。
|
||||
|
||||
## 当前目标
|
||||
|
||||
<!-- 手写：现在在做什么。插件不会碰这里。 -->
|
||||
|
||||
## 铁规则（常驻，永不衰减）
|
||||
|
||||
<!-- 手写：只写硬性要求。rules/ 目录下的文件会被自动常驻注入。 -->
|
||||
|
||||
## 目录登记
|
||||
|
||||
${AUTO_BEGIN} registry
|
||||
${AUTO_END}
|
||||
|
||||
## 加载路由表
|
||||
|
||||
${AUTO_BEGIN} routing
|
||||
${AUTO_END}
|
||||
`
|
||||
}
|
||||
|
||||
/** Create the skeleton (ROOT.md + category dirs) for a root that does not exist. */
|
||||
export function ensureRoot(root: MemoryRoot, categories: string[]): void {
|
||||
fs.mkdirSync(root.dir, { recursive: true })
|
||||
for (const category of categories) fs.mkdirSync(path.join(root.dir, category), { recursive: true })
|
||||
if (!fs.existsSync(path.join(root.dir, "ROOT.md"))) {
|
||||
fs.writeFileSync(path.join(root.dir, "ROOT.md"), defaultRootTemplate(root), "utf8")
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user