Masterpiece is a platform for a multimodal natural interface. We integrated Masterpiece into a new authoring tool for designers and engineers that uses 3D search capabilities to access original database content, supporting natural human-computer interaction. Masterpiece increases the user’s immersion with a physical interface that’s easier to use than a traditional mouse and keyboard. The user can generate and manipulate simple 3D objects with a sketch-based approach that integrates a multimodal gesture-speech interface. They can then assemble their 3D parts into more complex objects. Moreover, the user can access a database’s original 3D content using a 3D search engine. Using the rough sketch they created, users can search for similar 3D content in the database.

K. Moustakas, D. Tzovaras, S. Carbini, O. Bernier, J.E. Viallet, S. Raidt, M. Mancas, M. Dimiccoli, E. Yagci, S. Balci, E.I. Leon and M.G. Strintzis, “MASTERPIECE: Physical Interaction and 3D content-based search in VR Applications”, IEEE Multimedia Magazine, vol. 13, no. 3, pp. 92-100, July 2006.