Moses::Vocab Class Reference

Vocab maps between strings and uint32 ids. More...

#include <vocab.h>

Collaboration diagram for Moses::Vocab:

Collaboration graph
[legend]

List of all members.

Public Types

typedef std::map< Word, wordID_t > Word2Id
typedef std::map< wordID_t, Word > Id2Word

Public Member Functions

 Vocab (bool sntMarkers=true)
 Vocab (const std::string &vocab_path, const FactorDirection &direction, const FactorList &factors, bool closed=true)
 Vocab (FileHandler *fin, const FactorDirection &direction, const FactorList &factors, bool closed=true)
 Vocab (FileHandler *fin)
 ~Vocab ()
wordID_t GetWordID (const std::string &word, const FactorDirection &direction, const FactorList &factors, bool isNonTerminal)
wordID_t GetWordID (const Word &word)
wordID_t GetWordID (const std::string &word)
Word & GetWord (wordID_t id)
wordID_t GetkOOVWordID ()
wordID_t GetBOSWordID ()
const Word & GetkOOVWord ()
const Word & GetkBOSWord ()
const Word & GetkEOSWord ()
bool InVocab (wordID_t id)
bool InVocab (const Word &word)
uint32_t Size ()
void MakeClosed ()
void MakeOpen ()
bool IsClosed ()
bool Save (const std::string &vocab_path)
bool Save (FileHandler *fout)
bool Load (const std::string &vocab_path, const FactorDirection &direction, const FactorList &factors, bool closed=true)
bool Load (FileHandler *fin, const FactorDirection &direction, const FactorList &factors, bool closed=true)
bool Load (FileHandler *fin)
void PrintVocab ()
Word2Id::const_iterator VocabStart ()
Word2Id::const_iterator VocabEnd ()

Protected Member Functions

const Word InitSpecialWord (const std::string &type)
void InitSpecialWords ()

Protected Attributes

bool m_closed
const wordID_t m_kOOVWordID
const wordID_t m_kBOSWordID
Word m_kBOSWord
Word m_kEOSWord
Word m_kOOVWord
Word2Id m_words2ids
Id2Word m_ids2words

Detailed Description

Vocab maps between strings and uint32 ids.

Definition at line 16 of file vocab.h.

Member Typedef Documentation

typedef std::map<wordID_t, Word> Moses::Vocab::Id2Word

Definition at line 21 of file vocab.h.

typedef std::map<Word, wordID_t> Moses::Vocab::Word2Id

Definition at line 20 of file vocab.h.

Constructor & Destructor Documentation

Moses::Vocab::Vocab ( bool  sntMarkers = true  )  [inline]

Definition at line 23 of file vocab.h.

References GetWordID(), InitSpecialWords(), m_kBOSWord, and m_kEOSWord.

Here is the call graph for this function:

Moses::Vocab::Vocab ( const std::string &  vocab_path,
const FactorDirection &  direction,
const FactorList &  factors,
bool  closed = true 
) [inline]

Definition at line 36 of file vocab.h.

References InitSpecialWords(), Load(), and UTIL_THROW_IF2.

Here is the call graph for this function:

Moses::Vocab::Vocab ( FileHandler *  fin,
const FactorDirection &  direction,
const FactorList &  factors,
bool  closed = true 
) [inline]

Definition at line 44 of file vocab.h.

References InitSpecialWords(), Load(), and UTIL_THROW_IF2.

Here is the call graph for this function:

Moses::Vocab::Vocab ( FileHandler *  fin  )  [inline]

Definition at line 52 of file vocab.h.

References Load().

Here is the call graph for this function:

Moses::Vocab::~Vocab (  )  [inline]

Definition at line 57 of file vocab.h.

Member Function Documentation

wordID_t Moses::Vocab::GetBOSWordID (  )  [inline]

Definition at line 67 of file vocab.h.

References m_kBOSWordID.

Referenced by OnlineRLM< T >::insert().

Here is the caller graph for this function:

const Word& Moses::Vocab::GetkBOSWord (  )  [inline]

Definition at line 73 of file vocab.h.

References m_kBOSWord.

const Word& Moses::Vocab::GetkEOSWord (  )  [inline]

Definition at line 76 of file vocab.h.

References m_kEOSWord.

const Word& Moses::Vocab::GetkOOVWord (  )  [inline]

Definition at line 70 of file vocab.h.

References m_kOOVWord.

Referenced by Moses::BilingualDynSuffixArray::GetMosesFactorIDs(), and Load().

Here is the caller graph for this function:

wordID_t Moses::Vocab::GetkOOVWordID (  )  [inline]

Definition at line 64 of file vocab.h.

References m_kOOVWordID.

Referenced by OnlineRLM< T >::getProb().

Here is the caller graph for this function:

Word & Moses::Vocab::GetWord ( wordID_t  id  ) 

Definition at line 65 of file vocab.cpp.

References m_ids2words, and m_kOOVWord.

Referenced by Moses::BilingualDynSuffixArray::GetMosesFactorIDs().

Here is the caller graph for this function:

wordID_t Moses::Vocab::GetWordID ( const std::string &  word  ) 

Definition at line 29 of file vocab.cpp.

References Moses::Word::CreateFromString(), GetWordID(), and Moses::Input.

Here is the call graph for this function:

wordID_t Moses::Vocab::GetWordID ( const Word &  word  ) 

Definition at line 48 of file vocab.cpp.

References m_closed, m_ids2words, m_kOOVWordID, and m_words2ids.

wordID_t Moses::Vocab::GetWordID ( const std::string &  word,
const FactorDirection &  direction,
const FactorList &  factors,
bool  isNonTerminal 
)

Definition at line 39 of file vocab.cpp.

References Moses::Word::CreateFromString().

Referenced by GetWordID(), OnlineRLM< T >::insert(), OnlineRLM< T >::remove(), OnlineRLM< T >::sbsqQuery(), OnlineRLM< T >::update(), and Vocab().

Here is the call graph for this function:

Here is the caller graph for this function:

const Word Moses::Vocab::InitSpecialWord ( const std::string &  type  )  [protected]

Definition at line 15 of file vocab.cpp.

References Moses::Word::CreateFromString(), and Moses::Input.

Referenced by InitSpecialWords().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::Vocab::InitSpecialWords (  )  [protected]

Definition at line 8 of file vocab.cpp.

References BOS_, EOS_, InitSpecialWord(), m_kBOSWord, m_kEOSWord, m_kOOVWord, and UNKNOWN_FACTOR.

Referenced by Vocab().

Here is the call graph for this function:

Here is the caller graph for this function:

bool Moses::Vocab::InVocab ( const Word &  word  ) 

Definition at line 76 of file vocab.cpp.

References m_words2ids.

bool Moses::Vocab::InVocab ( wordID_t  id  ) 

Definition at line 71 of file vocab.cpp.

References m_ids2words.

bool Moses::Vocab::IsClosed (  )  [inline]

Definition at line 91 of file vocab.h.

References m_closed.

bool Moses::Vocab::Load ( FileHandler *  fin  ) 

Definition at line 106 of file vocab.cpp.

References Moses::Input, and Load().

Here is the call graph for this function:

bool Moses::Vocab::Load ( FileHandler *  fin,
const FactorDirection &  direction,
const FactorList &  factors,
bool  closed = true 
)

Definition at line 112 of file vocab.cpp.

References Moses::Word::CreateFromString(), GetkOOVWord(), m_closed, m_ids2words, m_words2ids, and UTIL_THROW_IF2.

Here is the call graph for this function:

bool Moses::Vocab::Load ( const std::string &  vocab_path,
const FactorDirection &  direction,
const FactorList &  factors,
bool  closed = true 
)

Definition at line 99 of file vocab.cpp.

Referenced by Load(), and Vocab().

Here is the caller graph for this function:

void Moses::Vocab::MakeClosed (  )  [inline]

Definition at line 85 of file vocab.h.

References m_closed.

void Moses::Vocab::MakeOpen (  )  [inline]

Definition at line 88 of file vocab.h.

References m_closed.

Referenced by OnlineRLM< T >::update().

Here is the caller graph for this function:

void Moses::Vocab::PrintVocab (  ) 

Definition at line 146 of file vocab.cpp.

References m_ids2words, and m_words2ids.

bool Moses::Vocab::Save ( FileHandler *  fout  ) 

Definition at line 88 of file vocab.cpp.

References m_ids2words.

bool Moses::Vocab::Save ( const std::string &  vocab_path  ) 

Definition at line 81 of file vocab.cpp.

Referenced by OnlineRLM< T >::save().

Here is the caller graph for this function:

uint32_t Moses::Vocab::Size (  )  [inline]

Definition at line 82 of file vocab.h.

References m_words2ids.

Referenced by OnlineRLM< T >::getProb().

Here is the caller graph for this function:

Word2Id::const_iterator Moses::Vocab::VocabEnd (  )  [inline]

Definition at line 105 of file vocab.h.

References m_words2ids.

Word2Id::const_iterator Moses::Vocab::VocabStart (  )  [inline]

Definition at line 102 of file vocab.h.

References m_words2ids.

Member Data Documentation

bool Moses::Vocab::m_closed [protected]

Definition at line 110 of file vocab.h.

Referenced by GetWordID(), IsClosed(), Load(), MakeClosed(), and MakeOpen().

Id2Word Moses::Vocab::m_ids2words [protected]

Definition at line 122 of file vocab.h.

Referenced by GetWord(), GetWordID(), InVocab(), Load(), PrintVocab(), and Save().

Word Moses::Vocab::m_kBOSWord [protected]

Definition at line 114 of file vocab.h.

Referenced by GetkBOSWord(), InitSpecialWords(), and Vocab().

const wordID_t Moses::Vocab::m_kBOSWordID [protected]

Definition at line 113 of file vocab.h.

Referenced by GetBOSWordID().

Word Moses::Vocab::m_kEOSWord [protected]

Definition at line 115 of file vocab.h.

Referenced by GetkEOSWord(), InitSpecialWords(), and Vocab().

Word Moses::Vocab::m_kOOVWord [protected]

Definition at line 116 of file vocab.h.

Referenced by GetkOOVWord(), GetWord(), and InitSpecialWords().

const wordID_t Moses::Vocab::m_kOOVWordID [protected]

Definition at line 112 of file vocab.h.

Referenced by GetkOOVWordID(), and GetWordID().

Word2Id Moses::Vocab::m_words2ids [protected]

Definition at line 121 of file vocab.h.

Referenced by GetWordID(), InVocab(), Load(), PrintVocab(), Size(), VocabEnd(), and VocabStart().

The documentation for this class was generated from the following files:
Generated on Mon Aug 17 00:34:07 2015 for Moses by  doxygen 1.5.9