Intention-Sensing Recipe Guidance via User Accessing to Objects

Abstract

Sensing the intention of a user’s forthcoming action is a necessary function for systems that assist human physical activity. In this article, a strategy for recipe guidance systems that can predict the forthcoming intended subtask in a cooking task is investigated. The focus is on user accessing objects, that is, touching and releasing objects. Touching can indicate the start of the forthcoming subtask and releasing can indicate the end of the task. The main difficulty lies in the fact that humans may move objects because they are in the way and use cooking tools that are unanticipated by an assistive system. In such cases, the accessed object should not indicate the forthcoming subtask. A method is proposed to track the progress of a task based on the object access history. This enables to eliminate object accesses that are out of context. Simultaneously, the method predicts the forthcoming subtask based on a combination of progress and materials rather than tools and materials. Then, a guidance system that runs as a web service is developed. In experiments, real cooking activities navigated by this system are observed. The Wizard of OZ method is utilized to simulate a system that detects object accesses. The experimental results show that 73.6% accuracy is achieved in the selection of the displayed information. This result supports the use of “access to objects” realize effective intention-sensing systems.