Moses::PhraseDictionaryOnDisk Class Reference

#include <PhraseDictionaryOnDisk.h>

Inheritance diagram for Moses::PhraseDictionaryOnDisk:

Collaboration diagram for Moses::PhraseDictionaryOnDisk:

List of all members.

Public Member Functions

PhraseDictionaryOnDisk (const std::string &line)

~PhraseDictionaryOnDisk ()

void Load (AllOptions::ptr const &opts)

override to load model files

virtual ChartRuleLookupManager * CreateRuleLookupManager (const ChartParser &parser, const ChartCellCollectionBase &, std::size_t)

Create a sentence-specific manager for SCFG rule lookup.

virtual void InitializeForInput (ttasksptr const &ttask)

Create entry for translation of source to targetPhrase.

void GetTargetPhraseCollectionBatch (const InputPathList &inputPathQueue) const

TargetPhraseCollection::shared_ptr GetTargetPhraseCollection (const OnDiskPt::PhraseNode *ptNode) const

TargetPhraseCollection::shared_ptr GetTargetPhraseCollectionNonCache (const OnDiskPt::PhraseNode *ptNode) const

Moses::TargetPhraseCollection::shared_ptr ConvertToMoses (const OnDiskPt::TargetPhraseCollection::shared_ptr targetPhrasesOnDisk, const std::vector< Moses::FactorType > &inputFactors, const std::vector< Moses::FactorType > &outputFactors, const Moses::PhraseDictionary &phraseDict, const std::vector< float > &weightT, OnDiskPt::Vocab &vocab, bool isSyntax) const

OnDiskPt::Word * ConvertFromMoses (OnDiskPt::OnDiskWrapper &wrapper, const std::vector< Moses::FactorType > &factorsVec, const Moses::Word &origWord) const

void SetParameter (const std::string &key, const std::string &value)

Protected Member Functions

OnDiskPt::OnDiskWrapper & GetImplementation ()

const OnDiskPt::OnDiskWrapper & GetImplementation () const

void GetTargetPhraseCollectionBatch (InputPath &inputPath) const

Moses::TargetPhrase * ConvertToMoses (const OnDiskPt::TargetPhrase &targetPhraseOnDisk, const std::vector< Moses::FactorType > &inputFactors, const std::vector< Moses::FactorType > &outputFactors, const OnDiskPt::Vocab &vocab, const Moses::PhraseDictionary &phraseDict, const std::vector< float > &weightT, bool isSyntax) const

void ConvertToMoses (const OnDiskPt::Word &wordOnDisk, const std::vector< Moses::FactorType > &outputFactorsVec, const OnDiskPt::Vocab &vocab, Moses::Word &overwrite) const

Protected Attributes

boost::scoped_ptr
< OnDiskPt::OnDiskWrapper > m_implementation

size_t m_maxSpanDefault

size_t m_maxSpanLabelled

Friends

class ChartRuleLookupManagerOnDisk

std::ostream & operator<< (std::ostream &, const PhraseDictionaryOnDisk &)

Detailed Description

Implementation of on-disk phrase table for hierarchical/syntax model.

Definition at line 47 of file PhraseDictionaryOnDisk.h.

Constructor & Destructor Documentation

Moses::PhraseDictionaryOnDisk::PhraseDictionaryOnDisk ( const std::string & line )

Definition at line 40 of file PhraseDictionaryOnDisk.cpp.

References Moses::FeatureFunction::ReadParameters().

Here is the call graph for this function:

Moses::PhraseDictionaryOnDisk::~PhraseDictionaryOnDisk ( )

Definition at line 48 of file PhraseDictionaryOnDisk.cpp.

Member Function Documentation

OnDiskPt::Word * Moses::PhraseDictionaryOnDisk::ConvertFromMoses	(	OnDiskPt::OnDiskWrapper &	wrapper,
		const std::vector< Moses::FactorType > &	factorsVec,
		const Moses::Word &	origWord
	)			const

Definition at line 347 of file PhraseDictionaryOnDisk.cpp.

References Moses::Word::GetFactor(), Moses::Factor::GetString(), Moses::Word::IsNonTerminal(), isNonTerminal(), NULL, util::StringStream::str(), and UTIL_THROW_IF2.

Referenced by Moses::ChartRuleLookupManagerOnDisk::GetChartRuleCollection(), and GetTargetPhraseCollectionBatch().

Here is the call graph for this function:

Here is the caller graph for this function:

Moses::TargetPhraseCollection::shared_ptr Moses::PhraseDictionaryOnDisk::ConvertToMoses	(	const OnDiskPt::TargetPhraseCollection::shared_ptr	targetPhrasesOnDisk,
		const std::vector< Moses::FactorType > &	inputFactors,
		const std::vector< Moses::FactorType > &	outputFactors,
		const Moses::PhraseDictionary &	phraseDict,
		const std::vector< float > &	weightT,
		OnDiskPt::Vocab &	vocab,
		bool	isSyntax
	)			const

Definition at line 220 of file PhraseDictionaryOnDisk.cpp.

References ConvertToMoses(), and Moses::PhraseDictionary::GetTableLimit().

Here is the call graph for this function:

void Moses::PhraseDictionaryOnDisk::ConvertToMoses	(	const OnDiskPt::Word &	wordOnDisk,
		const std::vector< Moses::FactorType > &	outputFactorsVec,
		const OnDiskPt::Vocab &	vocab,
		Moses::Word &	overwrite
	)			const `[protected]`

Definition at line 323 of file PhraseDictionaryOnDisk.cpp.

References Moses::FactorCollection::AddFactor(), Moses::Word::GetString(), Moses::FactorCollection::Instance(), Moses::Word::SetFactor(), and UTIL_THROW_IF2.

Here is the call graph for this function:

Moses::TargetPhrase * Moses::PhraseDictionaryOnDisk::ConvertToMoses	(	const OnDiskPt::TargetPhrase &	targetPhraseOnDisk,
		const std::vector< Moses::FactorType > &	inputFactors,
		const std::vector< Moses::FactorType > &	outputFactors,
		const OnDiskPt::Vocab &	vocab,
		const Moses::PhraseDictionary &	phraseDict,
		const std::vector< float > &	weightT,
		bool	isSyntax
	)			const `[protected]`

Definition at line 253 of file PhraseDictionaryOnDisk.cpp.

References Moses::Phrase::AddWord(), Moses::ScoreComponentCollection::Assign(), Moses::TargetPhrase::EvaluateInIsolation(), Moses::PhraseDictionary::GetFeaturesToApply(), Moses::TargetPhrase::GetScoreBreakdown(), Moses::Input, Moses::TargetPhrase::SetAlignNonTerm(), Moses::TargetPhrase::SetAlignTerm(), Moses::TargetPhrase::SetProperties(), Moses::TargetPhrase::SetTargetLHS(), and UTIL_THROW_IF2.

Referenced by ConvertToMoses(), Moses::ChartRuleLookupManagerOnDisk::GetChartRuleCollection(), and GetTargetPhraseCollectionNonCache().

Here is the call graph for this function:

Here is the caller graph for this function:

ChartRuleLookupManager * Moses::PhraseDictionaryOnDisk::CreateRuleLookupManager	(	const ChartParser &	,
		const ChartCellCollectionBase &	,
		std::size_t
	)			`[virtual]`

Create a sentence-specific manager for SCFG rule lookup.

Implements Moses::PhraseDictionary.

Definition at line 58 of file PhraseDictionaryOnDisk.cpp.

References ChartRuleLookupManagerOnDisk, GetImplementation(), Moses::DecodeFeature::m_input, and Moses::DecodeFeature::m_output.

Here is the call graph for this function:

const OnDiskPt::OnDiskWrapper & Moses::PhraseDictionaryOnDisk::GetImplementation ( ) const [protected]

Definition at line 77 of file PhraseDictionaryOnDisk.cpp.

References m_implementation, NULL, and UTIL_THROW_IF2.

OnDiskPt::OnDiskWrapper & Moses::PhraseDictionaryOnDisk::GetImplementation ( ) [protected]

Definition at line 69 of file PhraseDictionaryOnDisk.cpp.

References m_implementation, NULL, and UTIL_THROW_IF2.

Referenced by CreateRuleLookupManager(), GetTargetPhraseCollectionBatch(), and GetTargetPhraseCollectionNonCache().

Here is the caller graph for this function:

TargetPhraseCollection::shared_ptr Moses::PhraseDictionaryOnDisk::GetTargetPhraseCollection ( const OnDiskPt::PhraseNode * ptNode ) const

Definition at line 172 of file PhraseDictionaryOnDisk.cpp.

References Moses::PhraseDictionary::GetCache(), GetTargetPhraseCollectionNonCache(), and hash.

Referenced by GetTargetPhraseCollectionBatch().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::PhraseDictionaryOnDisk::GetTargetPhraseCollectionBatch ( const InputPathList & inputPathQueue ) const [virtual]

Reimplemented from Moses::PhraseDictionary.

Definition at line 112 of file PhraseDictionaryOnDisk.cpp.

References Moses::InputPath::GetPtNode(), and GetTargetPhraseCollectionBatch().

Here is the call graph for this function:

void Moses::PhraseDictionaryOnDisk::GetTargetPhraseCollectionBatch ( InputPath & inputPath ) const [protected]

Definition at line 129 of file PhraseDictionaryOnDisk.cpp.

References ConvertFromMoses(), GetImplementation(), Moses::InputPath::GetPhrase(), Moses::InputPath::GetPrevPath(), Moses::InputPath::GetPtNode(), Moses::Phrase::GetSize(), GetTargetPhraseCollection(), Moses::Phrase::GetWord(), Moses::DecodeFeature::m_input, Moses::DecodeFeature::m_inputFactors, NULL, Moses::Word::OnlyTheseFactors(), Moses::PhraseDictionary::SatisfyBackoff(), and Moses::InputPath::SetTargetPhrases().

Referenced by GetTargetPhraseCollectionBatch().

Here is the call graph for this function:

Here is the caller graph for this function:

TargetPhraseCollection::shared_ptr Moses::PhraseDictionaryOnDisk::GetTargetPhraseCollectionNonCache ( const OnDiskPt::PhraseNode * ptNode ) const

Definition at line 200 of file PhraseDictionaryOnDisk.cpp.

References ConvertToMoses(), GetImplementation(), Moses::StaticData::GetWeights(), Moses::StaticData::Instance(), Moses::DecodeFeature::m_input, Moses::DecodeFeature::m_output, and Moses::PhraseDictionary::m_tableLimit.

Referenced by GetTargetPhraseCollection().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::PhraseDictionaryOnDisk::InitializeForInput ( ttasksptr const & ttask ) [virtual]

Create entry for translation of source to targetPhrase.

Reimplemented from Moses::PhraseDictionary.

Definition at line 85 of file PhraseDictionaryOnDisk.cpp.

References Moses::PhraseDictionary::m_filePath, m_implementation, Moses::DecodeFeature::m_input, Moses::FeatureFunction::m_numScoreComponents, Moses::DecodeFeature::m_output, Moses::PhraseDictionary::ReduceCache(), and UTIL_THROW_IF2.

Here is the call graph for this function:

void Moses::PhraseDictionaryOnDisk::Load ( AllOptions::ptr const & opts ) [virtual]

override to load model files

Reimplemented from Moses::FeatureFunction.

Definition at line 52 of file PhraseDictionaryOnDisk.cpp.

References Moses::FeatureFunction::m_options, and Moses::PhraseDictionary::SetFeaturesToApply().

Here is the call graph for this function:

void Moses::PhraseDictionaryOnDisk::SetParameter	(	const std::string &	key,
		const std::string &	value
	)			`[virtual]`

Reimplemented from Moses::PhraseDictionary.

Definition at line 385 of file PhraseDictionaryOnDisk.cpp.

References m_maxSpanDefault, and m_maxSpanLabelled.

Friends And Related Function Documentation

friend class ChartRuleLookupManagerOnDisk [friend]

Definition at line 51 of file PhraseDictionaryOnDisk.h.

Referenced by CreateRuleLookupManager().

std::ostream& operator<<	(	std::ostream &	,
		const PhraseDictionaryOnDisk &
	)			`[friend]`

Member Data Documentation

boost::scoped_ptr<OnDiskPt::OnDiskWrapper> Moses::PhraseDictionaryOnDisk::m_implementation [protected]

Definition at line 57 of file PhraseDictionaryOnDisk.h.

Referenced by GetImplementation(), and InitializeForInput().

size_t Moses::PhraseDictionaryOnDisk::m_maxSpanDefault [protected]

Definition at line 60 of file PhraseDictionaryOnDisk.h.

Referenced by Moses::ChartRuleLookupManagerOnDisk::GetChartRuleCollection(), and SetParameter().

size_t Moses::PhraseDictionaryOnDisk::m_maxSpanLabelled [protected]

Definition at line 60 of file PhraseDictionaryOnDisk.h.

Referenced by Moses::ChartRuleLookupManagerOnDisk::GetChartRuleCollection(), and SetParameter().

The documentation for this class was generated from the following files:

/disk4/html/www/moses/doxygen/mosesdecoder/moses/TranslationModel/RuleTable/PhraseDictionaryOnDisk.h
/disk4/html/www/moses/doxygen/mosesdecoder/moses/TranslationModel/RuleTable/PhraseDictionaryOnDisk.cpp


Public Member Functions
	PhraseDictionaryOnDisk (const std::string &line)
	~PhraseDictionaryOnDisk ()
void	Load (AllOptions::ptr const &opts)
	override to load model files
virtual ChartRuleLookupManager *	CreateRuleLookupManager (const ChartParser &parser, const ChartCellCollectionBase &, std::size_t)
	Create a sentence-specific manager for SCFG rule lookup.
virtual void	InitializeForInput (ttasksptr const &ttask)
	Create entry for translation of source to targetPhrase.
void	GetTargetPhraseCollectionBatch (const InputPathList &inputPathQueue) const
TargetPhraseCollection::shared_ptr	GetTargetPhraseCollection (const OnDiskPt::PhraseNode *ptNode) const
TargetPhraseCollection::shared_ptr	GetTargetPhraseCollectionNonCache (const OnDiskPt::PhraseNode *ptNode) const
Moses::TargetPhraseCollection::shared_ptr	ConvertToMoses (const OnDiskPt::TargetPhraseCollection::shared_ptr targetPhrasesOnDisk, const std::vector< Moses::FactorType > &inputFactors, const std::vector< Moses::FactorType > &outputFactors, const Moses::PhraseDictionary &phraseDict, const std::vector< float > &weightT, OnDiskPt::Vocab &vocab, bool isSyntax) const
OnDiskPt::Word *	ConvertFromMoses (OnDiskPt::OnDiskWrapper &wrapper, const std::vector< Moses::FactorType > &factorsVec, const Moses::Word &origWord) const
void	SetParameter (const std::string &key, const std::string &value)
Protected Member Functions
OnDiskPt::OnDiskWrapper &	GetImplementation ()
const OnDiskPt::OnDiskWrapper &	GetImplementation () const
void	GetTargetPhraseCollectionBatch (InputPath &inputPath) const
Moses::TargetPhrase *	ConvertToMoses (const OnDiskPt::TargetPhrase &targetPhraseOnDisk, const std::vector< Moses::FactorType > &inputFactors, const std::vector< Moses::FactorType > &outputFactors, const OnDiskPt::Vocab &vocab, const Moses::PhraseDictionary &phraseDict, const std::vector< float > &weightT, bool isSyntax) const
void	ConvertToMoses (const OnDiskPt::Word &wordOnDisk, const std::vector< Moses::FactorType > &outputFactorsVec, const OnDiskPt::Vocab &vocab, Moses::Word &overwrite) const
Protected Attributes
boost::scoped_ptr < OnDiskPt::OnDiskWrapper >	m_implementation
size_t	m_maxSpanDefault
size_t	m_maxSpanLabelled
Friends
class	ChartRuleLookupManagerOnDisk
std::ostream &	operator<< (std::ostream &, const PhraseDictionaryOnDisk &)