Smart Brevity 7: Score the net, not the gross: memory, training data, and benchmarks judged against the plain baseline

Podcast: Smart Brevity · Jamiepluscoffee

Opublikowano: 2026-10-31

Score the net, not the gross: memory, training data, and benchmarks judged against the plain baseline; Memory tax: agent memory must beat naive RAG over raw history, and often does not.; Benchmarks as difficulty measures: measure mismatch and coverage gaps hide what a score really says.; Decision grounding: train only on actions the agent could justify from what it saw.; Turn-level credit assignment: agents decide in turns, so RL should assign credit per turn.; Interface lock-in: fine-tuning one answer format improves only that readout.; Decision models as routing economics: send yes/no agent steps to cheap typed-answer models.; In-situ interventions...