Accepted Papers


  1. NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks [Poster]

  2. Universality of Winning Tickets: A Renormalization Group Perspective [Poster]

  3. Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks [Poster] [Code]

  4. Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts [Poster] [Blog]

  5. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness [Poster] [Code]

  6. DRAGONN: Distributed Randomized Approximate Gradients of Neural Networks [Poster]

Honorable Mentions

  1. Learning to Merge Tokens in Vision Transformers [Poster]

  2. Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets [Poster]

  3. Lottery Tickets in Evolutionary Optimization: On Sparse Backpropagation-Free Trainability [Poster]

  4. The Unreasonable Effectiveness of Random Pruning: Return of the most naive baseline for sparse training [Poster] [Code]

  5. The State of Sparse Training in Deep Reinforcement Learning [Poster] [Code]


  1. APP: Anytime Progressive Pruning [Poster] [Code]

  2. Meta-Learning Sparse Compression Networks [Poster]

  3. The Price of Sparsity: Generalization and Memorization in Sparse Neural Network [Poster] [Code]

  4. Training Your Sparse Neural Network Better with Any Mask [Poster]

  5. Training Thinner and Deeper Neural Networks: Jumpstart Regularization [Poster]

  6. Bit-wise Training of Neural Network Weights [Poster]

  7. LidarCSNet: A Deep Convolutional Compressive Sensing Reconstruction Framework for 3D Airborne Lidar Point Cloud [Poster] [Blog]

  8. Adversarial robustness of sparse local Lipschitz predictors [Poster]

  9. From Hardness to Efficiency in Sparse Deep Network Training [Poster]

  10. Efficient Processing of Sparse and Compact DNN Models on Hardware Accelerators [Poster] [tweet]

  11. On the Presence of Winning Tickets in Model-Free Reinforcement Learning [Poster]

  12. A Brain-inspired Algorithm for Training Highly Sparse Neural Networks [Poster] [Code]

  13. Avoiding Forgetting and Allowing Forward Transfer in Continual Learning via Sparse Networks [Poster]

  14. Neural Implicit Dictionary Learning via Mixture-of-Expert Training [Poster] [Code]

  15. Training Recipe for N:M Structured Sparsity with Decaying Pruning Mask [Poster]

  16. Sparse*BERT: Sparse Models are Robust [Poster]

  17. Reverse-Engineering Sparse ReLU Networks [Poster]

  18. EGRU: Event-based GRU for activity-sparse inference and learning [Poster]

  19. Covid-19 Segmentation of the Lungs using a Sparse AE-CNN [Poster]

  20. Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training [Poster]

  21. Efficient identification of sparse neural networks with butterfly structure [Poster]

  22. Avoiding Catastrophe: Active Dendrites and Sparse Representations Enable Dynamic Multi-Task Learning [Poster] [Video]

  23. FasterAI: A Ligthweight Library for Creating Sparse Neural Networks [Poster] [Code] [Documentation]

  24. On the Emergence of Sparse Activation in Trained Transformer Models [Poster]

  25. Look-ups are not (yet) all you need for deep learning inference [Poster]

  26. Pruning Early Exit Networks [Poster]

  27. Robust Training under Label Noise by Over-parameterization [Poster]

  28. STen: An Interface for Efficient Sparsity in PyTorch [Poster] [Code]

  29. The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networks [Poster]

  30. Super Seeds: extreme model compression by trading off storage with computation [Poster]

  31. Recall Distortion in Neural Network Pruning and the Undecayed Pruning Algorithm [Poster]

  32. BERT Pruning: Is Magnitude All You Need? [Poster]

  33. S4 : a High-Sparsity, High-Performance AI Accelerator [Poster]

  34. Think Fast: Time Control in Varying Paradigms of Spiking Neural Networks [Poster]

  35. L0onie: Compressing COINs with L0-constraints [Poster] [Code]

  36. Pruning deep equilibrium models [Poster]

  37. Structural Learning in Artificial Neural Networks: A Neural Operator Perspective [Poster]

  38. Zeroth-Order Topological Insights into Iterative Magnitude Pruning [Poster]

  39. Analyzing the Confidentiality of Undistillable Teachers in Knowledge Distillation [Poster]

  40. Towards Low-Latency Energy-Efficient Deep SNNs via Attention-Guided Compression [Poster]

  41. The State of Unstructured Sparsity for Vision Transformers [Poster]

  42. Towards Implementing Truly Sparse Connections in Deep RL Agents [Poster]

  43. On the Robustness and Anomaly Detection of Sparse Neural Networks [Poster]

  44. Experimental implementation of a neural network optical channel equalizer in restricted hardware using pruning and quantization [Poster]

  45. Pruning Complex-Valued Neural Networks forOptical Channel Non-linear Impairments Mitigation [Poster]

  46. Low Rank Pruning via Output Perturbation [Poster]

  47. CrAM: A Compression-Aware Minimizer [Poster]

  48. Condensing Sparse Layers [Poster]

  49. Accelerating Sparse Training via Variance Reduction [Poster]

  50. Renormalized Sparse Neural Network Pruning [Poster]

  51. Sparse Probabilistic Circuits via Pruning and Growing [Poster]

  52. Weight-space ensembling of functionally diverse minima: Where does it all go wrong? [Poster]

  53. Finding Structured Winning Tickets with Early Kernel Pruning [Poster]

  54. Studying the impact of magnitude pruning on contrastive learning methods [Poster]