lm::ngram::SortedVocabulary Class Reference

#include <vocab.hh>

Inheritance diagram for lm::ngram::SortedVocabulary:

[legend]

Collaboration diagram for lm::ngram::SortedVocabulary:

[legend]

List of all members.

Public Member Functions
	SortedVocabulary ()
WordIndex	Index (const StringPiece &str) const
WordIndex	Bound () const
void	SetupMemory (void *start, std::size_t allocated, std::size_t entries, const Config &config)
void	Relocate (void *new_start)
void	ConfigureEnumerate (EnumerateVocab *to, std::size_t max_entries)
WordIndex	Insert (const StringPiece &str)
void	FinishedLoading (ProbBackoff *reorder_vocab)
std::size_t	UnkCountChangePadding () const
bool	SawUnk () const
void	LoadedBinary (bool have_words, int fd, EnumerateVocab *to, uint64_t offset)
uint64_t *&	EndHack ()
void	Populated ()
Static Public Member Functions
static uint64_t	Size (uint64_t entries, const Config &config)
static void	ComputeRenumbering (WordIndex types, int from_words, int to_words, std::vector< WordIndex > &mapping)

---

Detailed Description

Definition at line 67 of file vocab.hh.

---

Constructor & Destructor Documentation

lm::ngram::SortedVocabulary::SortedVocabulary

(

)

Definition at line 100 of file vocab.cc.

---

Member Function Documentation

WordIndex lm::ngram::SortedVocabulary::Bound

(

)

const [inline]

Definition at line 94 of file vocab.hh.

void lm::ngram::SortedVocabulary::ComputeRenumbering	(	WordIndex	types,
		int	from_words,
		int	to_words,
		std::vector< WordIndex > &	mapping
	)			[static]

Definition at line 164 of file vocab.cc.

References util::scoped_memory::get(), lm::ngram::detail::HashForVocab(), util::MapRead(), util::POPULATE_OR_READ, util::scoped_memory::reset(), StringPiece::size(), util::SizeOrThrow(), sort(), str, UTIL_THROW_IF, and UTIL_THROW_IF2.

Here is the call graph for this function:

void lm::ngram::SortedVocabulary::ConfigureEnumerate	(	EnumerateVocab *	to,
		std::size_t	max_entries
	)

Definition at line 121 of file vocab.cc.

References lm::EnumerateVocab::Add().

Here is the call graph for this function:

uint64_t*& lm::ngram::SortedVocabulary::EndHack

(

)

[inline]

Definition at line 115 of file vocab.hh.

void lm::ngram::SortedVocabulary::FinishedLoading

(

ProbBackoff *

reorder_vocab

)

Definition at line 146 of file vocab.cc.

WordIndex lm::ngram::SortedVocabulary::Index

(

const StringPiece &

str

)

const [inline, virtual]

Implements lm::base::Vocabulary.

Definition at line 71 of file vocab.hh.

References util::BoundedSortedUniformFind(), and lm::ngram::detail::HashForVocab().

Referenced by LoadedBinary(), and Populated().

Here is the call graph for this function:

Here is the caller graph for this function:

WordIndex lm::ngram::SortedVocabulary::Insert

(

const StringPiece &

str

)

Definition at line 129 of file vocab.cc.

References util::Pool::Allocate(), StringPiece::data(), lm::ngram::detail::HashForVocab(), and StringPiece::size().

Here is the call graph for this function:

void lm::ngram::SortedVocabulary::LoadedBinary	(	bool	have_words,
		int	fd,
		EnumerateVocab *	to,
		uint64_t	offset
	)

Definition at line 207 of file vocab.cc.

References Index(), and lm::base::Vocabulary::SetSpecial().

Here is the call graph for this function:

void lm::ngram::SortedVocabulary::Populated

(

)

Definition at line 200 of file vocab.cc.

References Index(), and lm::base::Vocabulary::SetSpecial().

Here is the call graph for this function:

void lm::ngram::SortedVocabulary::Relocate

(

void *

new_start

)

Definition at line 115 of file vocab.cc.

Referenced by lm::ngram::trie::BuildTrie().

Here is the caller graph for this function:

bool lm::ngram::SortedVocabulary::SawUnk

(

)

const [inline]

Definition at line 111 of file vocab.hh.

Referenced by UnkCountChangePadding().

Here is the caller graph for this function:

void lm::ngram::SortedVocabulary::SetupMemory	(	void *	start,
		std::size_t	allocated,
		std::size_t	entries,
		const Config &	config
	)

Definition at line 107 of file vocab.cc.

References Size().

Here is the call graph for this function:

uint64_t lm::ngram::SortedVocabulary::Size	(	uint64_t	entries,
		const Config &	config
	)			[static]

Definition at line 102 of file vocab.cc.

Referenced by SetupMemory().

Here is the caller graph for this function:

std::size_t lm::ngram::SortedVocabulary::UnkCountChangePadding

(

)

const [inline]

Definition at line 109 of file vocab.hh.

References SawUnk().

Referenced by lm::ngram::trie::BuildTrie().

Here is the call graph for this function:

Here is the caller graph for this function:

---

The documentation for this class was generated from the following files:

• /disk4/html/www/moses/doxygen/mosesdecoder/lm/vocab.hh
• /disk4/html/www/moses/doxygen/mosesdecoder/lm/vocab.cc