Structured World Representations in Maze-Solving Transformers
Published in NeurIPS 2023 UniReps Workshop, 2023
Transformers trained to solve mazes form linear representations of maze structure, and acquire interpretable attention heads which facilitate path-following.
Ivanitskiy, M.I.*, Spies, A.F.*, Räuker, T.* et al. Structured World Representations in Maze-Solving Transformers. NeurIPS 2023 UniReps Workshop. [BIB] [PDF]