Configuration ranking
Shape, not just score
Capability fingerprints
Specialists
Who leads each capability?
All configurations × all capabilities
Capability map
One capability, ranked
Capability detail
Paper finding
Scaling changes the ranking—not only the score.
Across Qwen 3.5 backbones, the strongest methods at 27B are not the systems that lead at 0.8B.
Qwen 3.5 · 0.8B to 27B
Mean accuracy trajectories
Where capacity helps
Axis-level gain
Head to head
Capability profile
A minus B
Capability deltas
Synthetic patients23
Clinical visits15,049
Dialogue turns≈296K
Median history14.7years
History length150K–954Ktokens
Benchmark design
Incremental memory under chronic-care pressure
MEME-bench evaluates how memory systems retain and use patient information across long-term, multi-department clinical records. It includes histories for 23 synthetic patients, with a total of 15,049 visits and approximately 296,000 dialogue turns. Individual histories contain 434–1,718 visits, span a median of 14.7 years, and range from 150,000 to 954,000 tokens.
Each memory system ingests a patient history over time and answers questions at predefined visit checkpoints using only the history available so far. The paper analyzes every system as a four-stage pipeline: curation, organization, retrieval, and utilization. In this Arena, a configuration is one answering backbone paired with one memory method.
Public release boundary
Aggregate results only
This Space contains no patient histories, raw questions, private preprocessing code, or credentials. Scores are average accuracy across three runs, graded by GPT-5.6-Sol, as reported in the paper draft.
The eight capabilities