On the dynamics of loss functions (Symmetry and Singularity of Geometric Structures and Differential Equations)

Abstract

In differential machine learning, one uses a stochastic gradient flow with respect to a loss function on the parameter space to find an "almost minimal'point of the loss function, which would correspond to an "almost optimal" predictor. Somehow, a proper theory of loss functions is still missing, despite their importance. Our paper is a contribution towards the construction of such a theory.