关于层出不穷的各式 AI memory system 的一些思考：我们应该把更多的精力放在设计更好的 eval 上，从而让最强的 memory system 进化出来
https://linghao.io/posts/memory-systems-should-be-evolved
https://linghao.io/posts/memory-systems-should-be-evolved
linghao.io
Evolving Memory Systems: An Eval-First Approach
AI memory systems are often designed as architectural bets: vector stores, profiles, summaries, graphs, etc. This post argues for a different starting point: build longitudinal eval environments where systems without good memory cannot survive, then let better…
🤔4
https://linghao.io/posts/taxonomy-differences-matter
以前觉得 taxonomy 只是无聊的分类学，开始做 LLM quality 以后几乎每天都在思考和跟同事争论 loss taxonomy 的细枝末节。这几天放假闲下来仔细想想，taxonomy 无处不在，至关重要。
以前觉得 taxonomy 只是无聊的分类学，开始做 LLM quality 以后几乎每天都在思考和跟同事争论 loss taxonomy 的细枝末节。这几天放假闲下来仔细想想，taxonomy 无处不在，至关重要。
linghao.io
A Taxonomy Is a Theory of What Differences Matter
Taxonomies are not neutral filing systems. They predict, coordinate, allocate, and govern. The right way to judge one depends on the job it is doing.
https://alexzhang13.github.io/blog/2026/harness/
> A good harness is a harness that reduces unfamiliar problems to familiar ones and reduces complex problems to simple ones. In other words, even if the state s is out-of-distribution (OOD) to what any individual language model call was trained for, a good harness produces observations o that are locally in-distribution (LID), which we define as every individual LM call over this observation being in-distribution with respect to the training data.
> A good harness is a harness that reduces unfamiliar problems to familiar ones and reduces complex problems to simple ones. In other words, even if the state s is out-of-distribution (OOD) to what any individual language model call was trained for, a good harness produces observations o that are locally in-distribution (LID), which we define as every individual LM call over this observation being in-distribution with respect to the training data.
Alex L. Zhang
Language model harnesses are compositional generalizers
Modern post-training brute-forces ever more environments and ever longer horizons, because frontier Transformers are still poor at compositional generalization. A good harness makes every model call locally in-distribution, so what it learns carries across…
👀1