MemT-1.0 Memory-Native LLMs Claim 66x KV Storage Reduction

A new paper, MemT-1.0: Towards Memory-Native Evolution, has been posted on Hugging Face. It introduces two models: MemT-1.0-Nano, with 36 billion parameters and 3 billion active, and MemT-1.0-Flash, with 288 billion parameters and 17 billion active. The paper reports 10 million and 336 million of dynamic memory for the two models respectively.

The backbones are Qwen3.6-35B-A3B for Nano and DeepSeek-V4-Flash for Flash. The core mechanism is a Gated DeltaNet-2 layer with token-level memory updates, plus a Mixture-of-Storage mechanism to scale native memory capacity.

The Claims

At 128K tokens of history, MemT-1.0-Nano reports a 1.86x end-to-end inference speedup and a 66x reduction in persistent storage compared with carrying a full KV cache.

On 11 memory benchmarks, MemT-1.0-Flash averages 50.6, which the paper says is 12.5 points above the best parametric baseline. On agentic tasks, the models gain 7.6 points on SWE-bench Verified and 8.3 points on SWE-bench Multilingual over their respective backbones.

What the Abstract Leaves Out

The abstract defers author affiliations to Appendix A.1. It also does not break the 12.5-point benchmark lead down by individual test.

Analysis

Building memory into the model itself, instead of carrying a full KV cache, is a different approach from many recent long-context methods. If the 66x storage reduction holds up under independent replication, it could meaningfully lower the cost of serving models with long conversation histories. For now, all of these numbers are self-reported by the paper, and the per-benchmark breakdown is what readers will want to see before drawing conclusions.

Pneumetron