edu.cmu.sphinx.linguist.dictionary

Class TextDictionary

java.lang.Object

edu.cmu.sphinx.linguist.dictionary.TextDictionary

All Implemented Interfaces:

Dictionary, Configurable

Direct Known Subclasses:

MappingDictionary, TrainerDictionary

public class TextDictionary
extends java.lang.Object
implements Dictionary

Creates a dictionary by quickly reading in an ASCII-based Sphinx-3 format dictionary. When loaded the dictionary just loads each line of the dictionary into the hash table, assuming that most words are not going to be used. Only when a word is actually used is its pronunciations massaged into an array of pronunciations.

The format of the ASCII dictionary is the word, followed by spaces or tab, followed by the pronunciation(s). For example, a digits dictionary will look like:

  ONE HH W AH N
  ONE(2) W AH N
  TWO T UW
  THREE TH R IY
  FOUR F AO R
  FIVE F AY V
  SIX S IH K S
  SEVEN S EH V AH N
  EIGHT EY T
  NINE N AY N
  ZERO Z IH R OW
  ZERO(2) Z IY R OW
  OH OW
 

In the above example, the words "one" and "zero" have two pronunciations each.

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.List<java.net.URL>	addendaUrlList
protected boolean	allocated
protected java.util.Map<java.lang.String,java.lang.String>	dictionary
protected static java.lang.String	FILLER_TAG
protected java.net.URL	fillerDictionaryFile
protected java.util.Set<java.lang.String>	fillerWords
protected G2PConverter	g2pDecoder
protected int	g2pMaxPron
protected java.net.URL	g2pModelFile
protected java.util.logging.Logger	logger
protected UnitManager	unitManager
protected java.util.Map<java.lang.String,Word>	wordDictionary
protected java.net.URL	wordDictionaryFile

Fields inherited from interface edu.cmu.sphinx.linguist.dictionary.Dictionary

PROP_ADDENDA, PROP_DICTIONARY, PROP_FILLER_DICTIONARY, PROP_G2P_MAX_PRONUNCIATIONS, PROP_G2P_MODEL_PATH, PROP_UNIT_MANAGER, PROP_WORD_REPLACEMENT, SENTENCE_END_SPELLING, SENTENCE_START_SPELLING, SILENCE_SPELLING

Constructor Summary

Constructors

Constructor and Description
TextDictionary()
TextDictionary(java.lang.String wordDictionaryFile, java.lang.String fillerDictionaryFile, java.util.List<java.net.URL> addendaUrlList, boolean addSilEndingPronunciation, java.lang.String wordReplacement, UnitManager unitManager)
TextDictionary(java.net.URL wordDictionaryFile, java.net.URL fillerDictionaryFile, java.util.List<java.net.URL> addendaUrlList, boolean addSilEndingPronunciation, java.lang.String wordReplacement, UnitManager unitManager, java.net.URL g2pModelFile, int g2pMaxPron)
TextDictionary(java.net.URL wordDictionaryFile, java.net.URL fillerDictionaryFile, java.util.List<java.net.URL> addendaUrlList, java.lang.String wordReplacement, UnitManager unitManager)

Method Summary

Modifier and Type	Method and Description
void	allocate() Allocates the dictionary
void	deallocate() Deallocates the dictionary
void	dump() Dumps this FastDictionary to System.out.
protected Unit	getCIUnit(java.lang.String name, boolean isFiller) Gets a context independent unit.
java.net.URL	getFillerDictionaryFile() Get the filler dictionary file
Word[]	getFillerWords() Gets the set of all filler words in the dictionary
Word	getSentenceEndWord() Returns the sentence end word.
Word	getSentenceStartWord() Returns the sentence start word.
Word	getSilenceWord() Returns the silence word.
Word	getWord(java.lang.String text) Returns a Word object based on the spelling and its classification.
java.net.URL	getWordDictionaryFile() Get the word dictionary file
protected void	loadDictionary(java.io.InputStream inputStream, boolean isFillerDict) Loads the given simple dictionary from the given InputStream.
void	newProperties(PropertySheet ps) This method is called when this configurable component needs to be reconfigured.
java.lang.String	toString() Returns a string representation of this TextDictionary in alphabetical order.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

logger

protected java.util.logging.Logger logger

wordDictionaryFile

protected java.net.URL wordDictionaryFile

fillerDictionaryFile

protected java.net.URL fillerDictionaryFile

addendaUrlList

protected java.util.List<java.net.URL> addendaUrlList

g2pModelFile

protected java.net.URL g2pModelFile

g2pMaxPron

protected int g2pMaxPron

unitManager

protected UnitManager unitManager

dictionary

protected java.util.Map<java.lang.String,java.lang.String> dictionary

wordDictionary

protected java.util.Map<java.lang.String,Word> wordDictionary

g2pDecoder

protected G2PConverter g2pDecoder

FILLER_TAG

protected static final java.lang.String FILLER_TAG

See Also:

Constant Field Values

fillerWords

protected java.util.Set<java.lang.String> fillerWords

allocated

protected boolean allocated

Constructor Detail

TextDictionary

public TextDictionary(java.lang.String wordDictionaryFile,
                      java.lang.String fillerDictionaryFile,
                      java.util.List<java.net.URL> addendaUrlList,
                      boolean addSilEndingPronunciation,
                      java.lang.String wordReplacement,
                      UnitManager unitManager)
               throws java.net.MalformedURLException,
                      java.lang.ClassNotFoundException

Throws:

java.net.MalformedURLException

java.lang.ClassNotFoundException

TextDictionary

public TextDictionary(java.net.URL wordDictionaryFile,
                      java.net.URL fillerDictionaryFile,
                      java.util.List<java.net.URL> addendaUrlList,
                      java.lang.String wordReplacement,
                      UnitManager unitManager)

TextDictionary

public TextDictionary(java.net.URL wordDictionaryFile,
                      java.net.URL fillerDictionaryFile,
                      java.util.List<java.net.URL> addendaUrlList,
                      boolean addSilEndingPronunciation,
                      java.lang.String wordReplacement,
                      UnitManager unitManager,
                      java.net.URL g2pModelFile,
                      int g2pMaxPron)

TextDictionary

public TextDictionary()

Method Detail

newProperties

public void newProperties(PropertySheet ps)
                   throws PropertyException

Description copied from interface: Configurable

This method is called when this configurable component needs to be reconfigured.

Specified by:

newProperties in interface Configurable

Parameters:

ps - a property sheet holding the new data

Throws:

PropertyException - if there is a problem with the properties.

getWordDictionaryFile

public java.net.URL getWordDictionaryFile()

Get the word dictionary file

Returns:

the URL of the word dictionary file

getFillerDictionaryFile

public java.net.URL getFillerDictionaryFile()

Get the filler dictionary file

Returns:

the URL of the filler dictionary file

allocate

public void allocate()
              throws java.io.IOException

Description copied from interface: Dictionary

Allocates the dictionary

Specified by:

allocate in interface Dictionary

Throws:

java.io.IOException - if there is trouble loading the dictionary

deallocate

public void deallocate()

Description copied from interface: Dictionary

Deallocates the dictionary

Specified by:

deallocate in interface Dictionary

loadDictionary

protected void loadDictionary(java.io.InputStream inputStream,
                              boolean isFillerDict)
                       throws java.io.IOException

Loads the given simple dictionary from the given InputStream. The InputStream is assumed to contain ASCII data.

Parameters:

inputStream - the InputStream of the dictionary

isFillerDict - true if this is a filler dictionary, false otherwise

Throws:

java.io.IOException - if there is an error reading the dictionary

getCIUnit

protected Unit getCIUnit(java.lang.String name,
                         boolean isFiller)

Gets a context independent unit. There should only be one instance of any CI unit

Parameters:

name - the name of the unit

isFiller - if true, the unit is a filler unit

Returns:

the unit

getSentenceStartWord

public Word getSentenceStartWord()

Returns the sentence start word.

Specified by:

getSentenceStartWord in interface Dictionary

Returns:

the sentence start word

getSentenceEndWord

public Word getSentenceEndWord()

Returns the sentence end word.

Specified by:

getSentenceEndWord in interface Dictionary

Returns:

the sentence end word

getSilenceWord

public Word getSilenceWord()

Returns the silence word.

Specified by:

getSilenceWord in interface Dictionary

Returns:

the silence word

getWord

public Word getWord(java.lang.String text)

Returns a Word object based on the spelling and its classification. The behavior of this method is also affected by the properties wordReplacement and g2pModel

Specified by:

getWord in interface Dictionary

Parameters:

text - the spelling of the word of interest.

Returns:

a Word object

See Also:

Word

toString

public java.lang.String toString()

Returns a string representation of this TextDictionary in alphabetical order.

Overrides:

toString in class java.lang.Object

Returns:

a string representation of this dictionary

getFillerWords

public Word[] getFillerWords()

Gets the set of all filler words in the dictionary

Specified by:

getFillerWords in interface Dictionary

Returns:

an array (possibly empty) of all filler words

dump

public void dump()

Dumps this FastDictionary to System.out.