UPM SEMINAR SERIES

Analog-Specific Algorithms for Efficient Neural Network Training 

Deep learning's rapid growth across industries is increasingly constrained by escalating energy demands for model training. Analog computing, especially using resistive memory devices, offers a potential breakthrough by enabling more efficient, parallel in-memory computation. 

Despite their potential, these devices exhibit inherent analog-specific characteristics, including noise, device-to-device and cycle-to-cycle variability, and asymmetric modulation responses. These non-idealities challenge traditional deep learning training paradigms, as straightforward implementations of stochastic gradient descent (SGD) converge inexactly, while adaptive optimizers like Adam are incompatible with hardware constraints. 

In this work, we explore how analog-specific algorithms can address these challenges, developing novel batching strategies that leverage the parallel computing capabilities of resistive memory devices. Using IBM's Analog Hardware Acceleration Kit, we simulate and validate these approaches, providing insights into their potential for efficient neural network training. 

Speaker: Ignacio Jiménez Gallo (visiting student conducting research at Microsystems Technology Laboratories (MTL) within the School of Engineering at Massachusetts Institute of Technology (MIT). 

LAB: Advanced Semiconductor Devices.

Devices for Machine Deep Learning 

As part of a research group developing analog memory devices for storing AI learning weight updates, our work focuses on identifying materials that meet the requirements to closely approximate the ideal model—a device with symmetric response, nanosecond-scale operation, 1V potential range, and a broad conductance spectrum to accurately represent different states. 

Our specific contribution lies in doping the cathode material, which serves as the key section for modifications and where the weight updates are physically stored. The goal is to identify a dopant that increases the base material’s resistance while allowing us to separate the total channel resistance into two distinct contributions: grain resistance and grain boundary resistance. 

Speaker: Guzmán del Puerto Llorente (UPM fellow visiting student at Massachusetts Institute of Technology (MIT). 

LAB: Solid-state ionic-electronic materials and devices.