« The Multiple Paths to Frugality in Machine Learning » by Florence d’Alché-Buc
Abstract: The spectacular results of data-driven AI, aka (deep) Machine Learning, come at the price of a massive energy consumption and an outpouring of human and material resources, whether it be for data annotation and collection, training or testing. In the context of climate change and limited resources, the concept of frugality has gained prominence in Machine Learning. We will provide a tentative review of the most promising approaches to build ml tools that require fewer resources without sacrificing predictive accuracy. In particular, we will highlight two main trends, data-efficient approaches and model/algorithm-efficient methods. We will conclude by pointing out the urgent need for taking into account this issue together with other ethical concerns and drastically changing the way we do machine learning.

« The New ISO/IEC Standard for Neural Network Coding » by Wojciech Samek
Abstract: The novel standard for Neural Network Coding (NNC), recently issued by ISO/IEC MPEG 15938-17, achieves very high coding gains, compressing neural networks less than 5% in size without accuracy loss. The underlying NNC encoder technology includes parameter quantization, followed by efficient arithmetic coding, namely DeepCABAC. In addition, NNC also allows very flexible adaptations, such as signaling specific local scaling values, setting quantization parameters per tensor rather than per network and supporting specific parameter fusion operations. This talk will discuss the different coding tools implemented withing the standard, along with their theoretical underpinnings and limitations. Furthermore, we will present NNCodec, the first open source and standard-compliant implementation of the NNC standard. Finally, we will discuss extensions towards differential neural network coding (dNNC), i.e., new compression methods  specifically tailored for incremental neural network updates. We benchmark dNNC in multiple federated and split learning scenarios using a variety of NN models and data including vision transformers and large-scale ImageNet experiments, showing a reduction in the size of the NN updates to less than 1% and a reduction of the overall energy consumption required for communication in federated learning systems by up to 94%.