Token (sphinx4-core 5prealpha-SNAPSHOT API)

java.lang.Object
- edu.cmu.sphinx.decoder.search.Token

All Implemented Interfaces:

Scoreable, Data
```
public class Token
extends java.lang.Object
implements Scoreable
```
Represents a single state in the recognition trellis. Subclasses of a token are used to represent the various emitting state.
All scores are maintained in LogMath log base

Field Summary
- Fields inherited from interface edu.cmu.sphinx.decoder.scorer.Scoreable
  COMPARATOR

Constructor Summary

Constructors
Constructor and Description
`Token(SearchState state, long collectTime)` Creates the initial token with the given word history depth
`Token(Token predecessor, float logTotalScore, float logAcousticScore, float logInsertionScore, float logLanguageScore)` Creates a Token with the given acoustic and language scores and predecessor.
`Token(Token predecessor, SearchState state, float logTotalScore, float logInsertionScore, float logLanguageScore, long collectTime)` Internal constructor for a token.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`float[]`	`calculateComponentScore(Data feature)`
`float`	`calculateScore(Data feature)` Calculates a score against the given feature.
`void`	`dumpTokenPath()` dumps a branch of tokens
`void`	`dumpTokenPath(boolean includeHMMStates)` dumps a branch of tokens
`float`	`getAcousticScore()` Returns the acoustic score for this token (in logMath log base).
`long`	`getCollectTime()` Collect time is different from frame number because some frames might be skipped in silence detector
`Data`	`getData()` Returns the feature for this Token.
`float`	`getInsertionScore()` Returns the insertion score associated with this token.
`float`	`getLanguageScore()` Returns the language score associated with this token
`protected static java.text.DecimalFormat`	`getNumberFormat()` Return the DecimalFormat object for formatting the print out of numbers
`Token`	`getPredecessor()` Returns the predecessor for this token, or null if this token has no predecessors
`float`	`getScore()` Returns the score for the token.
`protected static java.text.DecimalFormat`	`getScoreFormat()` Return the DecimalFormat object for formatting the print out of scores.
`SearchState`	`getSearchState()` Returns the SearchState associated with this token
`Word`	`getWord()` Returns the word of this Token, the search state is a WordSearchState.
`java.lang.String`	`getWordPath()` Returns the string of words for this token, with embedded silences
`java.lang.String`	`getWordPath(boolean wantFiller, boolean wantPronunciations)` Returns the string of words leading up to this token.
`java.lang.String`	`getWordPathNoFiller()` Returns the string of words for this token, with no embedded filler words
`java.lang.String`	`getWordUnitPath()` Returns the string of words and units for this token, with embedded silences.
`boolean`	`isEmitting()` Determines if this token is associated with an emitting state.
`boolean`	`isFinal()` Determines if this token is associated with a final SentenceHMM state.
`boolean`	`isWord()` Determines if this token marks the end of a word
`float`	`normalizeScore(float maxLogScore)` Normalizes a previously calculated score
`void`	`setData(Data data)` Sets the feature for this Token.
`void`	`setScore(float logScore)` Sets the score for this token
`static void`	`showCount()` Shows the token count
`java.lang.String`	`toString()` Retrieves the string representation of this object
`void`	`update(Token predecessor, SearchState nextState, float logEntryScore, float insertionProbability, float languageProbability, long collectTime)`
`boolean`	`validate()` Determines if this branch is valid

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Constructor Detail
  - Token
```
public Token(Token predecessor,
             SearchState state,
             float logTotalScore,
             float logInsertionScore,
             float logLanguageScore,
             long collectTime)
```
    Internal constructor for a token. Used by classes Token, CombineToken, ParallelToken
    
    Parameters:
    
    predecessor - the predecessor for this token
    
    state - the SentenceHMMState associated with this token
    
    logTotalScore - the total entry score for this token (in LogMath log base)
    
    logInsertionScore - the insertion score associated with this token (in LogMath log base)
    
    logLanguageScore - the language score associated with this token (in LogMath log base)
    
    collectTime - the frame collection time
  - Token
```
public Token(SearchState state,
             long collectTime)
```
    Creates the initial token with the given word history depth
    
    Parameters:
    
    state - the SearchState associated with this token
    
    collectTime - collection time of this token
  - Token
```
public Token(Token predecessor,
             float logTotalScore,
             float logAcousticScore,
             float logInsertionScore,
             float logLanguageScore)
```
    Creates a Token with the given acoustic and language scores and predecessor.
    
    Parameters:
    
    predecessor - previous token
    
    logTotalScore - total score
    
    logAcousticScore - the log acoustic score
    
    logInsertionScore - the log insertion score
    
    logLanguageScore - the log language score
- Method Detail
  - getPredecessor
```
public Token getPredecessor()
```
    Returns the predecessor for this token, or null if this token has no predecessors
    
    Returns:
    
    the predecessor
  - getCollectTime
```
public long getCollectTime()
```
    Collect time is different from frame number because some frames might be skipped in silence detector
    
    Returns:
    
    collection time in milliseconds
  - setData
```
public void setData(Data data)
```
    Sets the feature for this Token.
    
    Parameters:
    
    data - features
  - getData
```
public Data getData()
```
    Returns the feature for this Token.
    
    Returns:
    
    the feature for this Token
  - getScore
```
public float getScore()
```
    Returns the score for the token. The score is a combination of language and acoustic scores
    
    Specified by:
    
    getScore in interface Scoreable
    
    Returns:
    
    the score of this frame (in logMath log base)
  - calculateScore
```
public float calculateScore(Data feature)
```
    Calculates a score against the given feature. The score can be retrieved with get score. The token will keep a reference to the scored feature-vector.
    
    Specified by:
    
    calculateScore in interface Scoreable
    
    Parameters:
    
    feature - the feature to be scored
    
    Returns:
    
    the score for the feature
  - calculateComponentScore
```
public float[] calculateComponentScore(Data feature)
```
  - normalizeScore
```
public float normalizeScore(float maxLogScore)
```
    Normalizes a previously calculated score
    
    Specified by:
    
    normalizeScore in interface Scoreable
    
    Parameters:
    
    maxLogScore - the score to normalize this score with
    
    Returns:
    
    the normalized score
  - setScore
```
public void setScore(float logScore)
```
    Sets the score for this token
    
    Parameters:
    
    logScore - the new score for the token (in logMath log base)
  - getLanguageScore
```
public float getLanguageScore()
```
    Returns the language score associated with this token
    
    Returns:
    
    the language score (in logMath log base)
  - getInsertionScore
```
public float getInsertionScore()
```
    Returns the insertion score associated with this token. Insertion score is the score of the transition between states. It might be transition score from the acoustic model, phone insertion score or word insertion probability from the linguist.
    
    Returns:
    
    the language score (in logMath log base)
  - getAcousticScore
```
public float getAcousticScore()
```
    Returns the acoustic score for this token (in logMath log base). Acoustic score is a sum of frame GMM.
    
    Returns:
    
    score
  - getSearchState
```
public SearchState getSearchState()
```
    Returns the SearchState associated with this token
    
    Returns:
    
    the searchState
  - isEmitting
```
public boolean isEmitting()
```
    Determines if this token is associated with an emitting state. An emitting state is a state that can be scored acoustically.
    
    Returns:
    
    true if this token is associated with an emitting state
  - isFinal
```
public boolean isFinal()
```
    Determines if this token is associated with a final SentenceHMM state.
    
    Returns:
    
    true if this token is associated with a final state
  - isWord
```
public boolean isWord()
```
    Determines if this token marks the end of a word
    
    Returns:
    
    true if this token marks the end of a word
  - toString
```
public java.lang.String toString()
```
    Retrieves the string representation of this object
    
    Overrides:
    
    toString in class java.lang.Object
    
    Returns:
    
    the string representation of this object
  - dumpTokenPath
```
public void dumpTokenPath()
```
    dumps a branch of tokens
  - dumpTokenPath
```
public void dumpTokenPath(boolean includeHMMStates)
```
    dumps a branch of tokens
    
    Parameters:
    
    includeHMMStates - if true include all sentence hmm states
  - getWordPath
```
public java.lang.String getWordPath(boolean wantFiller,
                                    boolean wantPronunciations)
```
    Returns the string of words leading up to this token.
    
    Parameters:
    
    wantFiller - if true, filler words are added
    
    wantPronunciations - if true append [ phoneme phoneme ... ] after each word
    
    Returns:
    
    the word path
  - getWordPathNoFiller
```
public java.lang.String getWordPathNoFiller()
```
    Returns the string of words for this token, with no embedded filler words
    
    Returns:
    
    the string of words
  - getWordPath
```
public java.lang.String getWordPath()
```
    Returns the string of words for this token, with embedded silences
    
    Returns:
    
    the string of words
  - getWordUnitPath
```
public java.lang.String getWordUnitPath()
```
    Returns the string of words and units for this token, with embedded silences.
    
    Returns:
    
    the string of words and units
  - getWord
```
public Word getWord()
```
    Returns the word of this Token, the search state is a WordSearchState. If the search state is not a WordSearchState, return null.
    
    Returns:
    
    the word of this Token, or null if this is not a word token
  - showCount
```
public static void showCount()
```
    Shows the token count
  - validate
```
public boolean validate()
```
    Determines if this branch is valid
    
    Returns:
    
    true if the token and its predecessors are valid
  - getScoreFormat
```
protected static java.text.DecimalFormat getScoreFormat()
```
    Return the DecimalFormat object for formatting the print out of scores.
    
    Returns:
    
    the DecimalFormat object for formatting score print outs
  - getNumberFormat
```
protected static java.text.DecimalFormat getNumberFormat()
```
    Return the DecimalFormat object for formatting the print out of numbers
    
    Returns:
    
    the DecimalFormat object for formatting number print outs
  - update
```
public void update(Token predecessor,
                   SearchState nextState,
                   float logEntryScore,
                   float insertionProbability,
                   float languageProbability,
                   long collectTime)
```

Class Token

Field Summary

Fields inherited from interface edu.cmu.sphinx.decoder.scorer.Scoreable

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

Token

Token

Token

Method Detail

getPredecessor

getCollectTime

setData

getData

getScore

calculateScore

calculateComponentScore

normalizeScore

setScore

getLanguageScore

getInsertionScore

getAcousticScore

getSearchState

isEmitting

isFinal

isWord

toString

dumpTokenPath

dumpTokenPath

getWordPath

getWordPathNoFiller

getWordPath

getWordUnitPath

getWord

showCount

validate

getScoreFormat

getNumberFormat

update