| Date | One-day workshop on Friday, July 10, 2026 |
| Location | Coex Convention & Exhibition Center, Seoul, South Korea |
| Submission | Deadline |
| Latest News | Follow us at @weightsymmetry on X/Twitter for the updates! |
Overview
Neural network weight spaces have a rich geometric structure, shaped by symmetries inherent to the architecture. Even the simplest MLP exhibits neuron permutation symmetries, meaning that swapping any two neurons in a layer along with their weights does not change the network function. Modern architectures introduce many more: attention mechanisms add continuous rotation symmetries, nonlinearities and normalization layers create scaling symmetries, and multi-head and mixture of experts architectures introduce permutation symmetries over heads and experts. Altogether, these symmetries shape the loss landscape and training dynamics as well as play an important role in model analysis, merging, and learning from model weights.
This workshop aims to deepen the fundamental understanding of weight-space symmetries and their effects, and to translate these insights into practical and scalable methods with diverse applications.
Topics of interest include:
- Characterizing weight-space symmetries across model architectures.
- Loss landscape structure, training dynamics, and symmetry-aware optimization.
- Linear mode connectivity and model merging.
- Analysis of model weights and weight-space learning.
- Other applications: compression, quantization, uncertainty quantification, model safety, etc.
Questions? Contact us at [email protected] or @weightsymmetry.