Test-Time Learning and Memory
This page tracks models that adapt, memorize, or optimize during inference rather than relying only on static weights.
Sources in this batch
- NVIDIA frames context as training data for models that learn at test time.
- Titans and MIRAS are presented as mechanisms for long-term memory.
- Several sources connect test-time memorization, attentional bias, retention, and online optimization.
- Nested Learning is framed by Google as a new paradigm for continual learning.
- The continual-learning problem is discussed explicitly as a major obstacle for persistent agents.
- Test-time compute scaling sources connect memory/adaptation to inference-time reasoning.
Research interest
The surprising idea is that the context window may become an active learning substrate, not just passive prompt storage. This blurs boundaries between training, retrieval, memory, and inference. For a CS researcher, the key question is whether these mechanisms produce stable, controllable adaptation or just better in-context pattern matching.
Open questions:
- What information should be written into transient state versus durable weights or external memory?
- Can test-time learning avoid catastrophic interference and prompt injection style failures?
- How do we evaluate a model that changes its effective state over a task horizon?