Image statistics for material perception

Abstract

For estimation of material properties, inverse optics is generally too difficult to solve. Human material perception seems to rely on image features that are correlated with the material property under natural viewing environments. The critical features often take the form of image statistics, because many material properties can be characterized by how they optically modulate the natural image statistics. For instance, a critical image statistic for surface wetness perception is enhanced color saturations, while that for subresolution fineness perception is reduced luminance contrasts. There are optical reasons these image features vary in correlation with physical material properties, as well as psychophysical evidence that human material perception does respond to the features. That the shape (skewness) of the luminance histogram strongly affects surface material (gloss) perception, while not surface shape perception, suggests that material and shape perceptions may rely on independent image features — material (surface reflectance) perception relies on the magnitude of luminance gradient, while shape perception relies on the order of luminance gradient. I also discuss the merit and demerit of image statistics in relation to mid-level perceptual features, and deep neural network features.