Accession Number : ADA310721

Title :   Scalable Trigram Backoff Language Models,

Corporate Author : CARNEGIE-MELLON UNIV PITTSBURGH PA DEPT OF COMPUTER SCIENCE

Personal Author(s) : Seymore, Kristie ; Rosenfeld, Ronald

PDF Url : ADA310721

Report Date : MAY 1996

Pagination or Media Count : 21

Abstract : When a trigram backoff language model is created from a large body of text, trigrams and bigrams that occur few times in the training text are often excluded from the model in order to decrease the model size. Generally, the elimination of n-grams with very low counts is believed to not significantly affect model performance. This project investigates the degradation of a trigram backoff model's perplexity and word error rates as bigram and trigram cutoffs are increased. The advantage of reduction in model size is compared to the increase in word error rate and perplexity scores. More importantly, this project also investigates alternative ways of excluding bigrams and trigrams from a backoff language model, using criteria other than the number of times an n-gram occurred in the training text. Specifically, a difference method has been investigated where the difference in the logs of the original and backed off trigram and bigram probabilities was used as a basis for n-gram exclusion from the model. We have shown that excluding trigrams and bigrams based on a weighted version of this difference method results in better perplexity and word error rate performance than excluding trigrams and bigrams based on counts alone.

Descriptors :   *MATHEMATICAL MODELS, *SPEECH RECOGNITION, *PATTERN RECOGNITION, DATA BASES, OPTIMIZATION, DATA MANAGEMENT, MAXIMUM LIKELIHOOD ESTIMATION, ERROR ANALYSIS, WORDS(LANGUAGE), SYSTEMS ANALYSIS, VOCABULARY, TEXT PROCESSING, DECODERS.

Subject Categories : Cybernetics

Distribution Statement : APPROVED FOR PUBLIC RELEASE