Benjamin Valpey, Drake University
Benjamin Valpey
Drake University
Understanding the Numerical Behavior of NVIDIA’s Tensor Cores
Join this talk via Zoom: Click to join
NVIDIA’s Tensor Cores accelerate matrix multiplication on GPUs by rapidly computing D = A*B + C for small floating-point matrices. Although they were designed for machine learning, their performance makes them attractive for many other domains. For accuracy-critical applications, however, there is a serious obstacle: NVIDIA does not document the numerical behavior of these units, and prior work has shown that they deviate from IEEE-754, the standard that defines how floating-point operations should behave. When even small numerical differences matter, this uncertainty makes Tensor Cores difficult to use with confidence. This talk presents a formal model of the numerical behavior of Tensor Cores. Building on this model, we present an error-free transformation (EFT) for Tensor Cores. An EFT computes the exact rounding error of a result, allowing the true value to be recovered. With the EFT, we demonstrate how to reclaim correctly rounded results from the values Tensor Cores compute.
