edu.cmu.sphinx.linguist.language.ngram.trie

Class NgramTrieModel

java.lang.Object

edu.cmu.sphinx.linguist.language.ngram.trie.NgramTrieModel

All Implemented Interfaces:

LanguageModel, Configurable

public class NgramTrieModel
extends java.lang.Object
implements LanguageModel

Language model that uses a binary NGram language model file ("binary trie file") generated by the SphinxBase sphinx_lm_convert.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	NgramTrieModel.TrieRange Structure to keep ngram indexes range for trie traversal
static class	NgramTrieModel.TrieUnigram Structure that keeps unigram instance data in trie.

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	applyLanguageWeightAndWip
protected boolean	clearCacheAfterUtterance
protected int[]	counts
protected int	curDepth
protected Dictionary	dictionary
protected java.lang.String	format
protected float	languageWeight
protected java.util.logging.Logger	logger
protected LogMath	logMath
protected float	logWip
protected int	maxDepth
protected int	ngramCacheSize
protected java.lang.String	ngramLogFile
static java.lang.String	PROP_APPLY_LANGUAGE_WEIGHT_AND_WIP The property that controls whether or not the language model will apply the language weight and word insertion probability
static java.lang.String	PROP_CLEAR_CACHES_AFTER_UTTERANCE The property that controls whether the ngram caches are cleared after every utterance
static java.lang.String	PROP_LANGUAGE_WEIGHT The property that defines the language weight for the search
static java.lang.String	PROP_NGRAM_CACHE_SIZE The property that defines that maximum number of ngrams to be cached
static java.lang.String	PROP_QUERY_LOG_FILE The property for the name of the file that logs all the queried N-grams.
static java.lang.String	PROP_WORD_INSERTION_PROBABILITY Word insertion probability property
protected NgramTrieQuant	quant
protected NgramTrie	trie
protected java.util.Map<Word,java.lang.Integer>	unigramIDMap
protected NgramTrieModel.TrieUnigram[]	unigrams
protected float	unigramWeight
protected java.lang.String[]	words

Fields inherited from interface edu.cmu.sphinx.linguist.language.ngram.LanguageModel

PROP_DICTIONARY, PROP_LOCATION, PROP_MAX_DEPTH, PROP_UNIGRAM_WEIGHT

Constructor Summary

Constructors

Constructor and Description
NgramTrieModel()
NgramTrieModel(java.lang.String format, java.net.URL location, java.lang.String ngramLogFile, int maxNGramCacheSize, boolean clearCacheAfterUtterance, int maxDepth, Dictionary dictionary, boolean applyLanguageWeightAndWip, float languageWeight, double wip, float unigramWeight)

Method Summary

Modifier and Type	Method and Description
void	allocate() Create the language model
void	deallocate() Deallocate resources allocated to this language model
int	getMaxDepth() Returns the maximum depth of the language model
int	getNGramHits() Returns the number of NGram hits.
int	getNGramMisses() Returns the number of times when a NGram is queried, but there is no such NGram in the LM (in which case it uses the backoff probabilities).
float	getProbability(WordSequence wordSequence) Gets the ngram probability of the word sequence represented by the word list
float	getSmear(WordSequence wordSequence) Gets the smear term for the given wordSequence
java.util.Set<java.lang.String>	getVocabulary() Returns the set of words in the language model.
void	newProperties(PropertySheet ps) This method is called when this configurable component needs to be reconfigured.
void	onUtteranceEnd() Called by lexicon after recognition.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PROP_QUERY_LOG_FILE

@S4String(mandatory=false)
public static final java.lang.String PROP_QUERY_LOG_FILE

The property for the name of the file that logs all the queried N-grams. If this property is set to null, it means that the queried N-grams are not logged.

See Also:

Constant Field Values

PROP_NGRAM_CACHE_SIZE

@S4Integer(defaultValue=100000)
public static final java.lang.String PROP_NGRAM_CACHE_SIZE

The property that defines that maximum number of ngrams to be cached

See Also:

Constant Field Values

PROP_CLEAR_CACHES_AFTER_UTTERANCE

@S4Boolean(defaultValue=false)
public static final java.lang.String PROP_CLEAR_CACHES_AFTER_UTTERANCE

The property that controls whether the ngram caches are cleared after every utterance

See Also:

Constant Field Values

PROP_LANGUAGE_WEIGHT

@S4Double(defaultValue=1.0)
public static final java.lang.String PROP_LANGUAGE_WEIGHT

The property that defines the language weight for the search

See Also:

Constant Field Values

PROP_APPLY_LANGUAGE_WEIGHT_AND_WIP

@S4Boolean(defaultValue=false)
public static final java.lang.String PROP_APPLY_LANGUAGE_WEIGHT_AND_WIP

The property that controls whether or not the language model will apply the language weight and word insertion probability

See Also:

Constant Field Values

PROP_WORD_INSERTION_PROBABILITY

@S4Double(defaultValue=1.0)
public static final java.lang.String PROP_WORD_INSERTION_PROBABILITY

Word insertion probability property

See Also:

Constant Field Values

logger

protected java.util.logging.Logger logger

logMath

protected LogMath logMath

maxDepth

protected int maxDepth

curDepth

protected int curDepth

counts

protected int[] counts

ngramCacheSize

protected int ngramCacheSize

clearCacheAfterUtterance

protected boolean clearCacheAfterUtterance

dictionary

protected Dictionary dictionary

format

protected java.lang.String format

applyLanguageWeightAndWip

protected boolean applyLanguageWeightAndWip

languageWeight

protected float languageWeight

unigramWeight

protected float unigramWeight

logWip

protected float logWip

ngramLogFile

protected java.lang.String ngramLogFile

unigrams

protected NgramTrieModel.TrieUnigram[] unigrams

words

protected java.lang.String[] words

quant

protected NgramTrieQuant quant

trie

protected NgramTrie trie

unigramIDMap

protected java.util.Map<Word,java.lang.Integer> unigramIDMap

Constructor Detail

NgramTrieModel

public NgramTrieModel(java.lang.String format,
                      java.net.URL location,
                      java.lang.String ngramLogFile,
                      int maxNGramCacheSize,
                      boolean clearCacheAfterUtterance,
                      int maxDepth,
                      Dictionary dictionary,
                      boolean applyLanguageWeightAndWip,
                      float languageWeight,
                      double wip,
                      float unigramWeight)

NgramTrieModel

public NgramTrieModel()

Method Detail

newProperties

public void newProperties(PropertySheet ps)
                   throws PropertyException

Description copied from interface: Configurable

This method is called when this configurable component needs to be reconfigured.

Specified by:

newProperties in interface Configurable

Parameters:

ps - a property sheet holding the new data

Throws:

PropertyException - if there is a problem with the properties.

allocate

public void allocate()
              throws java.io.IOException

Description copied from interface: LanguageModel

Create the language model

Specified by:

allocate in interface LanguageModel

Throws:

java.io.IOException - if error occurrs

deallocate

public void deallocate()
                throws java.io.IOException

Description copied from interface: LanguageModel

Deallocate resources allocated to this language model

Specified by:

deallocate in interface LanguageModel

Throws:

java.io.IOException - if error occurrs

getProbability

public float getProbability(WordSequence wordSequence)

Gets the ngram probability of the word sequence represented by the word list

Specified by:

getProbability in interface LanguageModel

Parameters:

wordSequence - - the word sequence

Returns:

the probability of the word sequence. Probability is in logMath log base

getSmear

public float getSmear(WordSequence wordSequence)

Gets the smear term for the given wordSequence

Specified by:

getSmear in interface LanguageModel

Parameters:

wordSequence - - the word sequence

Returns:

the smear term associated with this word sequence

getVocabulary

public java.util.Set<java.lang.String> getVocabulary()

Returns the set of words in the language model. The set is unmodifiable.

Specified by:

getVocabulary in interface LanguageModel

Returns:

the unmodifiable set of words

getNGramMisses

public int getNGramMisses()

Returns the number of times when a NGram is queried, but there is no such NGram in the LM (in which case it uses the backoff probabilities).

Returns:

the number of NGram misses

getNGramHits

public int getNGramHits()

Returns the number of NGram hits.

Returns:

the number of NGram hits

getMaxDepth

public int getMaxDepth()

Returns the maximum depth of the language model

Specified by:

getMaxDepth in interface LanguageModel

Returns:

the maximum depth of the language model

onUtteranceEnd

public void onUtteranceEnd()

Called by lexicon after recognition. Used to clear caches

Specified by:

onUtteranceEnd in interface LanguageModel