Moses::PhraseDictionary Class Reference

#include <PhraseDictionary.h>

Inheritance diagram for Moses::PhraseDictionary:

Collaboration diagram for Moses::PhraseDictionary:

List of all members.

Public Member Functions

virtual bool ProvidesPrefixCheck () const

PhraseDictionary (const std::string &line, bool registerNow)

virtual ~PhraseDictionary ()

size_t GetTableLimit () const

table limit number.

size_t GetId () const

continguous id for each pt, starting from 0

virtual bool PrefixExists (ttasksptr const &ttask, Phrase const &phrase) const

return true if phrase table entries starting with /phrase/

virtual
TargetPhraseCollection::shared_ptr GetTargetPhraseCollectionLEGACY (const Phrase &src) const

find list of translations that can translates src. Only for phrase input

virtual
TargetPhraseCollection::shared_ptr GetTargetPhraseCollectionLEGACY (ttasksptr const &ttask, Phrase const &src) const

virtual void GetTargetPhraseCollectionBatch (const InputPathList &inputPathQueue) const

virtual void GetTargetPhraseCollectionBatch (ttasksptr const &ttask, InputPathList const &inputPathQueue) const

virtual void InitializeForInput (ttasksptr const &ttask)

Create entry for translation of source to targetPhrase.

virtual void CleanUpAfterSentenceProcessing (const InputType &source)

virtual ChartRuleLookupManager * CreateRuleLookupManager (const ChartParser &, const ChartCellCollectionBase &, std::size_t)=0

Create a sentence-specific manager for SCFG rule lookup.

const std::string & GetFilePath () const

const std::vector
< FeatureFunction * > & GetFeaturesToApply () const

void SetParameter (const std::string &key, const std::string &value)

virtual
TargetPhraseCollectionWithSourcePhrase::shared_ptr GetTargetPhraseCollectionLEGACY (InputType const &src, Range const &range) const

find list of translations that can translates a portion of src. Used by confusion network decoding

Static Public Member Functions

static const std::vector
< PhraseDictionary * > & GetColl ()

Protected Member Functions

void SetFeaturesToApply ()

bool SatisfyBackoff (const InputPath &inputPath) const

virtual
TargetPhraseCollection::shared_ptr GetTargetPhraseCollectionNonCacheLEGACY (const Phrase &src) const

void ReduceCache () const

CacheColl & GetCache () const

Protected Attributes

size_t m_tableLimit

std::string m_filePath

std::vector< FeatureFunction * > m_featuresToApply

size_t m_maxCacheSize

boost::scoped_ptr< CacheColl > m_cache

size_t m_id

Static Protected Attributes

static std::vector
< PhraseDictionary * > s_staticColl

Friends

class PhraseDictionaryMultiModelCounts

Detailed Description

Abstract base class for phrase dictionaries (tables).

Definition at line 66 of file PhraseDictionary.h.

Constructor & Destructor Documentation

Moses::PhraseDictionary::PhraseDictionary	(	const std::string &	line,
		bool	registerNow
	)

Definition at line 38 of file PhraseDictionary.cpp.

References m_id, and s_staticColl.

virtual Moses::PhraseDictionary::~PhraseDictionary ( ) [inline, virtual]

Definition at line 80 of file PhraseDictionary.h.

Member Function Documentation

virtual void Moses::PhraseDictionary::CleanUpAfterSentenceProcessing ( const InputType & source ) [inline, virtual]

Reimplemented from Moses::FeatureFunction.

Reimplemented in Moses::PhraseDictionaryCompact, Moses::PhraseDictionaryGroup, Moses::PhraseDictionaryMultiModel, Moses::PhraseDictionaryTransliteration, Moses::PhraseDictionaryTreeAdaptor, Moses::PhraseDictionaryALSuffixArray, and Moses::PhraseDictionaryFuzzyMatch.

Definition at line 131 of file PhraseDictionary.h.

virtual ChartRuleLookupManager* Moses::PhraseDictionary::CreateRuleLookupManager	(	const ChartParser &	,
		const ChartCellCollectionBase &	,
		std::size_t
	)			`[pure virtual]`

Create a sentence-specific manager for SCFG rule lookup.

Referenced by Moses::ChartParser::ChartParser().

Here is the caller graph for this function:

CacheColl & Moses::PhraseDictionary::GetCache ( ) const [protected]

Definition at line 202 of file PhraseDictionary.cpp.

References m_cache, and NULL.

Referenced by Moses::PhraseDictionaryOnDisk::GetTargetPhraseCollection(), Moses::PhraseDictionaryTransliteration::GetTargetPhraseCollection(), Moses::ExamplePT::GetTargetPhraseCollectionBatch(), GetTargetPhraseCollectionLEGACY(), and ReduceCache().

Here is the caller graph for this function:

static const std::vector<PhraseDictionary*>& Moses::PhraseDictionary::GetColl ( ) [inline, static]

Definition at line 74 of file PhraseDictionary.h.

References s_staticColl.

Referenced by Moses::ChartParser::ChartParser(), Moses::StaticData::CheckLEGACYPT(), MosesServer::Updater::execute(), Moses::FindPhraseDictionary(), Moses::PhraseDictionaryGroup::Load(), Moses::InputFeature::Load(), Moses::StaticData::LoadFeatureFunctions(), Moses::StaticData::NoCache(), Moses::Manager::OutputWordGraph(), Moses::PDTAimp::PDTAimp(), Moses::ChartParserUnknown::Process(), Moses::ProcessAndStripXMLTags(), Moses::TranslationOptionCollection::ProcessOneUnknownWord(), Moses::TargetPhrase::SetXMLScore(), and Moses::TranslationOptionCollectionConfusionNet::TranslationOptionCollectionConfusionNet().

Here is the caller graph for this function:

const std::vector<FeatureFunction*>& Moses::PhraseDictionary::GetFeaturesToApply ( ) const [inline]

Definition at line 144 of file PhraseDictionary.h.

References m_featuresToApply.

Referenced by Moses::PhraseDictionaryOnDisk::ConvertToMoses(), Moses::ExamplePT::CreateTargetPhrase(), Moses::PDTAimp::CreateTargetPhrase(), Moses::ProbingPT::CreateTargetPhrases(), Moses::PhraseDecoder::DecodeCollection(), Moses::DecodeStepTranslation::DecodeStepTranslation(), Moses::PhraseDictionaryDynamicCacheBased::GetTargetPhraseCollection(), Moses::PhraseDictionaryFuzzyMatch::InitializeForInput(), Moses::PhraseDictionaryMemoryPerSentenceOnDemand::InitializeForInput(), Moses::PhraseDictionaryMemoryPerSentence::InitializeForInput(), Moses::RuleTableLoaderStandard::Load(), Moses::Syntax::T2S::RuleTrieLoader::Load(), Moses::Syntax::S2T::RuleTrieLoader::Load(), and Moses::Syntax::F2S::HyperTreeLoader::Load().

Here is the caller graph for this function:

const std::string& Moses::PhraseDictionary::GetFilePath ( ) const [inline]

Definition at line 140 of file PhraseDictionary.h.

References m_filePath.

Referenced by Moses::PhraseDictionaryALSuffixArray::InitializeForInput(), Moses::RuleTableLoaderStandard::Load(), Moses::Syntax::T2S::RuleTrieLoader::Load(), and Moses::Syntax::S2T::RuleTrieLoader::Load().

Here is the caller graph for this function:

size_t Moses::PhraseDictionary::GetId ( ) const [inline]

continguous id for each pt, starting from 0

Definition at line 89 of file PhraseDictionary.h.

References m_id.

Referenced by Moses::PhrasePenalty::EvaluateInIsolation().

Here is the caller graph for this function:

size_t Moses::PhraseDictionary::GetTableLimit ( ) const [inline]

table limit number.

Definition at line 84 of file PhraseDictionary.h.

References m_tableLimit.

Referenced by Moses::PhraseDictionaryOnDisk::ConvertToMoses(), Moses::ChartRuleLookupManagerOnDisk::GetChartRuleCollection(), Moses::Syntax::T2S::RuleTrieLoader::Load(), Moses::Syntax::S2T::RuleTrieLoader::Load(), Moses::Syntax::F2S::HyperTreeLoader::Load(), Moses::DecodeStepTranslation::Process(), Moses::DecodeStepTranslation::ProcessInitialTranslation(), Moses::DecodeStepTranslation::ProcessInitialTransLEGACY(), Moses::DecodeStepTranslation::ProcessLEGACY(), Moses::PhraseDictionaryFuzzyMatch::SortAndPrune(), and Moses::PhraseDictionaryMemory::SortAndPrune().

Here is the caller graph for this function:

virtual void Moses::PhraseDictionary::GetTargetPhraseCollectionBatch	(	ttasksptr const &	ttask,
		InputPathList const &	inputPathQueue
	)			const `[inline, virtual]`

Reimplemented in Moses::PhraseDictionaryGroup, and Moses::Mmsapt.

Definition at line 123 of file PhraseDictionary.h.

References GetTargetPhraseCollectionBatch().

Here is the call graph for this function:

void Moses::PhraseDictionary::GetTargetPhraseCollectionBatch ( const InputPathList & inputPathQueue ) const [virtual]

Reimplemented in Moses::ExamplePT, Moses::PhraseDictionaryMemory, Moses::PhraseDictionaryMemoryPerSentence, Moses::PhraseDictionaryMemoryPerSentenceOnDemand, Moses::PhraseDictionaryTransliteration, Moses::ProbingPT, and Moses::PhraseDictionaryOnDisk.

Definition at line 150 of file PhraseDictionary.cpp.

References Moses::InputPath::GetPhrase(), GetTargetPhraseCollectionLEGACY(), NULL, SatisfyBackoff(), and Moses::InputPath::SetTargetPhrases().

Referenced by Moses::TranslationOptionCollection::GetTargetPhraseCollectionBatch(), and GetTargetPhraseCollectionBatch().

Here is the call graph for this function:

Here is the caller graph for this function:

TargetPhraseCollectionWithSourcePhrase::shared_ptr Moses::PhraseDictionary::GetTargetPhraseCollectionLEGACY	(	InputType const &	src,
		Range const &	range
	)			const `[virtual]`

find list of translations that can translates a portion of src. Used by confusion network decoding

Reimplemented in Moses::PhraseDictionaryTreeAdaptor.

Definition at line 97 of file PhraseDictionary.cpp.

References UTIL_THROW.

virtual TargetPhraseCollection::shared_ptr Moses::PhraseDictionary::GetTargetPhraseCollectionLEGACY	(	ttasksptr const &	ttask,
		Phrase const &	src
	)			const `[inline, virtual]`

Reimplemented in Moses::PhraseDictionaryGroup, and Moses::Mmsapt.

Definition at line 113 of file PhraseDictionary.h.

References GetTargetPhraseCollectionLEGACY().

Here is the call graph for this function:

TargetPhraseCollection::shared_ptr Moses::PhraseDictionary::GetTargetPhraseCollectionLEGACY ( const Phrase & src ) const [virtual]

find list of translations that can translates src. Only for phrase input

Reimplemented in Moses::PhraseDictionaryDynamicCacheBased, Moses::PhraseDictionaryGroup, Moses::PhraseDictionaryMemory, Moses::PhraseDictionaryMultiModel, and Moses::PhraseDictionaryMultiModelCounts.

Definition at line 56 of file PhraseDictionary.cpp.

References GetCache(), GetTargetPhraseCollectionNonCacheLEGACY(), hash, Moses::hash_value(), and m_maxCacheSize.

Referenced by Moses::PhraseDictionaryMultiModel::CollectSufficientStatistics(), Moses::PhraseDictionaryGroup::CreateTargetPhraseCollection(), Moses::PhraseDictionaryMultiModelCounts::GetTargetCount(), GetTargetPhraseCollectionBatch(), GetTargetPhraseCollectionLEGACY(), main(), Moses::DecodeStepTranslation::ProcessInitialTransLEGACY(), and Moses::DecodeStepTranslation::ProcessLEGACY().

Here is the call graph for this function:

Here is the caller graph for this function:

TargetPhraseCollection::shared_ptr Moses::PhraseDictionary::GetTargetPhraseCollectionNonCacheLEGACY ( const Phrase & src ) const [protected, virtual]

Reimplemented in Moses::PhraseDictionaryCompact, Moses::PhraseDictionaryDynamicCacheBased, Moses::PhraseDictionaryMemoryPerSentenceOnDemand, and Moses::PhraseDictionaryTreeAdaptor.

Definition at line 89 of file PhraseDictionary.cpp.

References UTIL_THROW.

Referenced by GetTargetPhraseCollectionLEGACY().

Here is the caller graph for this function:

virtual void Moses::PhraseDictionary::InitializeForInput ( ttasksptr const & ttask ) [inline, virtual]

Create entry for translation of source to targetPhrase.

Reimplemented from Moses::FeatureFunction.

Reimplemented in Moses::ExamplePT, Moses::PhraseDictionaryDynamicCacheBased, Moses::PhraseDictionaryGroup, Moses::PhraseDictionaryMemoryPerSentence, Moses::PhraseDictionaryMemoryPerSentenceOnDemand, Moses::PhraseDictionaryMultiModel, Moses::PhraseDictionaryMultiModelCounts, Moses::PhraseDictionaryTreeAdaptor, Moses::ProbingPT, Moses::PhraseDictionaryALSuffixArray, Moses::PhraseDictionaryFuzzyMatch, Moses::PhraseDictionaryOnDisk, and Moses::Mmsapt.

Definition at line 128 of file PhraseDictionary.h.

Referenced by main().

Here is the caller graph for this function:

bool Moses::PhraseDictionary::PrefixExists	(	ttasksptr const &	ttask,
		Phrase const &	phrase
	)			const `[virtual]`

return true if phrase table entries starting with /phrase/

Reimplemented in Moses::Mmsapt.

Definition at line 143 of file PhraseDictionary.cpp.

Referenced by Moses::PhraseDictionaryGroup::GetTargetPhraseCollectionBatch().

Here is the caller graph for this function:

bool Moses::PhraseDictionary::ProvidesPrefixCheck ( ) const [virtual]

Reimplemented in Moses::Mmsapt.

Definition at line 49 of file PhraseDictionary.cpp.

Referenced by Moses::TranslationOptionCollectionConfusionNet::TranslationOptionCollectionConfusionNet().

Here is the caller graph for this function:

void Moses::PhraseDictionary::ReduceCache ( ) const [protected]

Definition at line 168 of file PhraseDictionary.cpp.

References GetCache(), m_maxCacheSize, Moses::Timer::start(), and VERBOSE.

Referenced by Moses::PhraseDictionaryTransliteration::CleanUpAfterSentenceProcessing(), Moses::PhraseDictionaryCompact::CleanUpAfterSentenceProcessing(), Moses::PhraseDictionaryOnDisk::InitializeForInput(), Moses::PhraseDictionaryTreeAdaptor::InitializeForInput(), Moses::PhraseDictionaryDynamicCacheBased::InitializeForInput(), and Moses::ExamplePT::InitializeForInput().

Here is the call graph for this function:

Here is the caller graph for this function:

bool Moses::PhraseDictionary::SatisfyBackoff ( const InputPath & inputPath ) const [protected]

Definition at line 214 of file PhraseDictionary.cpp.

References Moses::DecodeGraph::GetBackoff(), Moses::DecodeFeature::GetDecodeGraph(), Moses::InputPath::GetPhrase(), Moses::Phrase::GetSize(), Moses::InputPath::GetTargetPhrases(), and Moses::DecodeFeature::m_container.

Referenced by Moses::PhraseDictionaryOnDisk::GetTargetPhraseCollectionBatch(), Moses::PhraseDictionaryTransliteration::GetTargetPhraseCollectionBatch(), Moses::PhraseDictionaryMemory::GetTargetPhraseCollectionBatch(), and GetTargetPhraseCollectionBatch().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::PhraseDictionary::SetFeaturesToApply ( ) [protected]

Definition at line 119 of file PhraseDictionary.cpp.

References Moses::FeatureFunction::GetFeatureFunctions(), Moses::FeatureFunction::IsUseable(), m_featuresToApply, and Moses::DecodeFeature::m_outputFactors.

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::PhraseDictionary::SetParameter	(	const std::string &	key,
		const std::string &	value
	)			`[virtual]`

Reimplemented from Moses::DecodeFeature.

Reimplemented in Moses::PhraseDictionaryDynamicCacheBased, Moses::PhraseDictionaryGroup, Moses::PhraseDictionaryMemoryPerSentenceOnDemand, Moses::PhraseDictionaryMultiModel, Moses::PhraseDictionaryMultiModelCounts, Moses::PhraseDictionaryTransliteration, Moses::ProbingPT, Moses::PhraseDictionaryFuzzyMatch, and Moses::PhraseDictionaryOnDisk.

Definition at line 104 of file PhraseDictionary.cpp.

References m_filePath, m_maxCacheSize, and m_tableLimit.

Referenced by Moses::StaticData::NoCache().

Here is the caller graph for this function:

Friends And Related Function Documentation

friend class PhraseDictionaryMultiModelCounts [friend]

Definition at line 68 of file PhraseDictionary.h.

Member Data Documentation

boost::scoped_ptr<CacheColl> Moses::PhraseDictionary::m_cache [mutable, protected]

Definition at line 178 of file PhraseDictionary.h.

Referenced by GetCache().

std::vector<FeatureFunction*> Moses::PhraseDictionary::m_featuresToApply [protected]

Definition at line 164 of file PhraseDictionary.h.

Referenced by GetFeaturesToApply(), and SetFeaturesToApply().

std::string Moses::PhraseDictionary::m_filePath [protected]

Definition at line 160 of file PhraseDictionary.h.

Referenced by GetFilePath(), Moses::PhraseDictionaryTransliteration::GetTargetPhraseCollection(), Moses::PhraseDictionaryOnDisk::InitializeForInput(), Moses::PhraseDictionaryTreeAdaptor::InitializeForInput(), Moses::PhraseDictionaryMemoryPerSentence::InitializeForInput(), Moses::RuleTableTrie::Load(), Moses::ProbingPT::Load(), Moses::PhraseDictionaryCompact::Load(), Moses::Syntax::RuleTableFF::Load(), and SetParameter().

size_t Moses::PhraseDictionary::m_id [protected]

Definition at line 189 of file PhraseDictionary.h.

Referenced by GetId(), and PhraseDictionary().

size_t Moses::PhraseDictionary::m_maxCacheSize [protected]

Definition at line 172 of file PhraseDictionary.h.

Referenced by GetTargetPhraseCollectionLEGACY(), Moses::PhraseDictionaryDynamicCacheBased::PhraseDictionaryDynamicCacheBased(), Moses::PhraseDictionaryMemory::PhraseDictionaryMemory(), Moses::PhraseDictionaryScope3::PhraseDictionaryScope3(), ReduceCache(), Moses::Syntax::RuleTableFF::RuleTableFF(), and SetParameter().

size_t Moses::PhraseDictionary::m_tableLimit [protected]

Definition at line 159 of file PhraseDictionary.h.

Referenced by Moses::ProbingPT::CreateTargetPhrases(), GetTableLimit(), Moses::PhraseDictionaryDynamicCacheBased::GetTargetPhraseCollection(), Moses::PhraseDictionaryMultiModelCounts::GetTargetPhraseCollectionLEGACY(), Moses::PhraseDictionaryMultiModel::GetTargetPhraseCollectionLEGACY(), Moses::PhraseDictionaryGroup::GetTargetPhraseCollectionLEGACY(), Moses::PhraseDictionaryOnDisk::GetTargetPhraseCollectionNonCache(), Moses::PhraseDictionaryCompact::GetTargetPhraseCollectionNonCacheLEGACY(), Moses::PhraseDictionaryALSuffixArray::InitializeForInput(), Moses::RuleTableTrie::Load(), and SetParameter().

std::vector< PhraseDictionary * > Moses::PhraseDictionary::s_staticColl [static, protected]

< all the score producers in this run

Reimplemented from Moses::FeatureFunction.

Definition at line 157 of file PhraseDictionary.h.

Referenced by GetColl(), and PhraseDictionary().

The documentation for this class was generated from the following files:

/disk4/html/www/moses/doxygen/mosesdecoder/moses/TranslationModel/PhraseDictionary.h
/disk4/html/www/moses/doxygen/mosesdecoder/moses/TranslationModel/PhraseDictionary.cpp


Public Member Functions
virtual bool	ProvidesPrefixCheck () const
	PhraseDictionary (const std::string &line, bool registerNow)
virtual	~PhraseDictionary ()
size_t	GetTableLimit () const
	table limit number.
size_t	GetId () const
	continguous id for each pt, starting from 0
virtual bool	PrefixExists (ttasksptr const &ttask, Phrase const &phrase) const
	return true if phrase table entries starting with /phrase/
virtual TargetPhraseCollection::shared_ptr	GetTargetPhraseCollectionLEGACY (const Phrase &src) const
	find list of translations that can translates src. Only for phrase input
virtual TargetPhraseCollection::shared_ptr	GetTargetPhraseCollectionLEGACY (ttasksptr const &ttask, Phrase const &src) const
virtual void	GetTargetPhraseCollectionBatch (const InputPathList &inputPathQueue) const
virtual void	GetTargetPhraseCollectionBatch (ttasksptr const &ttask, InputPathList const &inputPathQueue) const
virtual void	InitializeForInput (ttasksptr const &ttask)
	Create entry for translation of source to targetPhrase.
virtual void	CleanUpAfterSentenceProcessing (const InputType &source)
virtual ChartRuleLookupManager *	CreateRuleLookupManager (const ChartParser &, const ChartCellCollectionBase &, std::size_t)=0
	Create a sentence-specific manager for SCFG rule lookup.
const std::string &	GetFilePath () const
const std::vector < FeatureFunction * > &	GetFeaturesToApply () const
void	SetParameter (const std::string &key, const std::string &value)
virtual TargetPhraseCollectionWithSourcePhrase::shared_ptr	GetTargetPhraseCollectionLEGACY (InputType const &src, Range const &range) const
	find list of translations that can translates a portion of src. Used by confusion network decoding
Static Public Member Functions
static const std::vector < PhraseDictionary * > &	GetColl ()
Protected Member Functions
void	SetFeaturesToApply ()
bool	SatisfyBackoff (const InputPath &inputPath) const
virtual TargetPhraseCollection::shared_ptr	GetTargetPhraseCollectionNonCacheLEGACY (const Phrase &src) const
void	ReduceCache () const
CacheColl &	GetCache () const
Protected Attributes
size_t	m_tableLimit
std::string	m_filePath
std::vector< FeatureFunction * >	m_featuresToApply
size_t	m_maxCacheSize
boost::scoped_ptr< CacheColl >	m_cache
size_t	m_id
Static Protected Attributes
static std::vector < PhraseDictionary * >	s_staticColl
Friends
class	PhraseDictionaryMultiModelCounts