Dynamic Projection Mapping on Multiple Non-Rigid Moving Objects for Stage Performance Applications

Abstract

In this paper we have proposed and developed a projection mapping system that can project 3D images on multiple moving objects that change their positions and shapes. Although projection mapping has become a new type of image projection method, so far most of the projection mappings have focused on the projection of 3D images on objects such as buildings that stay static without changing their shapes. If projection mapping on non-rigid moving objects such as human performers is realized, application area of projection mapping will become far wider. To demonstrate this, a simple test of integrating projection mapping and Noh performance was carried out. Although Noh performance has been considered difficult to understand for beginners and foreigners because of its minimalism, by projecting 2D images that suggest inner emotion of a main character and also ongoing story, the evaluation result show that this new type of Noh performance could make it easier to be understood. This means that the integration of projection mapping and various stage performances could open-up new possibilities for various kinds of performing arts to add their values. To realize this, we have proposed a projection mapping system consisting of multiple depth sensors and multiple projectors. In the first step, using multiple depth sensors the position and shape of multiple objects are detected in real time and the obtained 3D models of the multiple objects are sent to the projection phase. In the projection phase, the images to be projected on the objects are rendered on the obtained 3D model of the object. Then using multiple projectors real-time 3D projection mapping is carried out on the multiple moving objects. The system is now under development and demonstration of actual Noh performance using this technology will be achieved.