The Digital Press

Transformers traditionally designed for flat images struggle with spherical data common in physics and robotics. This work develops a spherical attention mechanism that respects the sphere's geometry, improving accuracy and efficiency for such tasks.

Why it matters: Preserving spherical symmetries ensures physically accurate modeling in climate science, cosmology, and robotics.

The big picture: Integrating quadrature weights yields rotationally equivariant attention, enhancing Transformer performance on spherical domains.

Stunning stat: Spherical Transformers outperform planar models on three diverse tasks, including fluid simulation and spherical image segmentation.

Quick takeaway: Neighborhood attention reduces complexity and enforces locality while maintaining spherical symmetry for scalable, efficient models.