Using Modality Replacement to Facilitate Communication between Visually and Hearing-Impaired People

Our proposed framework consists of a situated communication environment designed to foster an immersive experience for the visually and hearing impaired. In situated communication, the partners interact in a real-time, shared environment.9 Physical modalities, such as speech rather than haptic text, are used for the visually impaired, and sign language, rather than graphic text, is used for the hearing impaired because speech and sign language, unlike text, are situated communication modalities. Concerning the immersive experience of the environment and its objects, consistent object-to-object modality transformations are used instead of object-to-object-property transformations.