Accession Number : ADA336561

Title :   Modeling and Interpreting Multimodal Inputs: A Semantic Integration Approach

Corporate Author : CARNEGIE-MELLON UNIV PITTSBURGH PA SCHOOL OF COMPUTER SCIENCE

Personal Author(s) : Vo, Minh T. ; Waibel, Alex

PDF Url : ADA336561

Report Date : DEC 1997

Pagination or Media Count : 22

Abstract : Modern user interfaces can take advantage of multiple input modalities such as speech, gestures, handwriting... to increase robustness and flexibility. The construction of such multimodal interfaces would be greatly facilitated by a unified framework that provides methods to characterize and interpret multimodal inputs. In this paper we describe a semantic model and a multimodal grammar structure for a broad class of multimodal applications. We also present a set of grammar-based Java tools that facilitate the construction of multimodal input processing modules, including a connectionist network for multimodal semantic integration.

Descriptors :   *SEMANTICS, *SPEECH RECOGNITION, ALGORITHMS, COMPUTER COMMUNICATIONS, HIGH LEVEL LANGUAGES, SYNTAX, CONTEXT FREE GRAMMARS, OBJECT ORIENTED PROGRAMMING, GRAPHICAL USER INTERFACE.

Subject Categories : Computer Programming and Software

Distribution Statement : APPROVED FOR PUBLIC RELEASE