Transformer Key-Value Memories Are Nearly as Interpretable as Sparse Autoencoders

Ye, Mengyu; Suzuki, Jun; Inaba, Tatsuro; Kuribayashi, Tatsuki

Transformer Key-Value Memories Are Nearly as Interpretable as Sparse Autoencoders

Mengyu Ye¹, Jun Suzuki^1,2, Tatsuro Inaba^1,3, Tatsuki Kuribayashi³

¹Tohoku University, ²RIKEN, ³MBZUAI
NeurIPS 2025

Paper Code arXiv

Aliquam vitae elit ullamcorper tellus egestas pellentesque. Ut lacus tellus, maximus vel lectus at, placerat pretium mi. Maecenas dignissim tincidunt vestibulum. Sed consequat hendrerit nisl ut maximus.

Abstract

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin ullamcorper tellus sed ante aliquam tempus. Etiam porttitor urna feugiat nibh elementum, et tempor dolor mattis. Donec accumsan enim augue, a vulputate nisi sodales sit amet. Proin bibendum ex eget mauris cursus euismod nec et nibh. Maecenas ac gravida ante, nec cursus dui. Vivamus purus nibh, placerat ac purus eget, sagittis vestibulum metus. Sed vestibulum bibendum lectus gravida commodo. Pellentesque auctor leo vitae sagittis suscipit.

First image description.

Second image description.

Third image description.

Fourth image description.

Video Presentation

Another Carousel

Poster

BibTeX

@article{YourPaperKey2024,
  title={Your Paper Title Here},
  author={First Author and Second Author and Third Author},
  journal={Conference/Journal Name},
  year={2024},
  url={https://your-domain.com/your-project-page}
}