Moses::TranslationOptionCollection Class Reference

#include <TranslationOptionCollection.h>

Inheritance diagram for Moses::TranslationOptionCollection:

Inheritance graph
[legend]
Collaboration diagram for Moses::TranslationOptionCollection:

Collaboration graph
[legend]

List of all members.

Public Member Functions

TranslationOptionListGetTranslationOptionList (size_t startPos, size_t endPos)
 list of trans opt for a particular span
TranslationOptionList const * GetTranslationOptionList (size_t startPos, size_t endPos) const
virtual ~TranslationOptionCollection ()
const InputTypeGetSource () const
 input sentence/confusion network
const std::vector< const
Phrase * > & 
GetUnknownSources () const
 List of unknowns (OOVs).
virtual void CreateTranslationOptions ()
 Create all possible translations from the phrase tables.
virtual bool CreateTranslationOptionsForRange (const DecodeGraph &decodeStepList, size_t startPosition, size_t endPosition, bool adhereTableLimit, size_t graphInd)=0
 Create translation options that exactly cover a specific input span.
virtual bool HasXmlOptionsOverlappingRange (size_t startPosition, size_t endPosition) const
 Check if this range has XML options.
virtual bool ViolatesXmlOptionsConstraint (size_t startPosition, size_t endPosition, TranslationOption *transOpt) const
 Check if a subsumed XML option constraint is satisfied.
virtual void CreateXmlOptionsForRange (size_t startPosition, size_t endPosition)
 Create xml-based translation options for the specific input span.
virtual const SquareMatrixGetEstimatedScores () const
 returns future cost matrix for sentence
TranslationOptionList const * GetTranslationOptionList (const Range &coverage) const
 list of trans opt for a particular span
const InputPathListGetInputPaths () const
ttasksptr GetTranslationTask () const
 TO_STRING ()

Protected Member Functions

 TranslationOptionCollection (ttasksptr const &ttask, InputType const &src)
void CalcEstimatedScore ()
void ProcessUnknownWord ()
 Force a creation of a translation option where there are none for a particular source position.
virtual void ProcessOneUnknownWord (const InputPath &inputPath, size_t sourcePos, size_t length=1, const ScorePair *inputScores=NULL)
 special handling of ONE unknown words.
void Prune ()
 pruning: only keep the top n (m_maxNoTransOptPerCoverage) elements */
void Sort ()
 sort all trans opt in each list for cube pruning */
void Add (TranslationOption *translationOption)
virtual void ProcessUnknownWord (size_t sourcePos)=0
 implemented by inherited class, called by this class
void EvaluateWithSourceContext ()
void EvaluateTranslationOptionListWithSourceContext (TranslationOptionList &)
void CacheLexReordering ()
void GetTargetPhraseCollectionBatch ()
bool CreateTranslationOptionsForRange (const DecodeGraph &decodeGraph, size_t startPos, size_t endPos, bool adhereTableLimit, size_t graphInd, InputPath &inputPath)
void SetInputScore (const InputPath &inputPath, PartialTranslOptColl &oldPtoc)

Protected Attributes

ttaskwptr m_ttask
std::vector< std::vector
< TranslationOptionList > > 
m_collection
InputType const & m_source
SquareMatrix m_estimatedScores
const size_t m_maxNoTransOptPerCoverage
const float m_translationOptionThreshold
size_t m_max_phrase_length
size_t max_partial_trans_opt
std::vector< const Phrase * > m_unksrcs
InputPathList m_inputPathQueue

Friends

std::ostream & operator<< (std::ostream &out, const TranslationOptionCollection &coll)


Detailed Description

Contains all phrase translations applicable to current input type (a sentence or confusion network). A key insight into efficient decoding is that various input conditions (trelliss, factored input, normal text, xml markup) all lead to the same decoding algorithm: hypotheses are expanded by applying phrase translations, which can be precomputed.

The precomputation of a collection of instances of such TranslationOption depends on the input condition, but they all are presented to decoding algorithm in the same form, using this class.

This is a abstract class, and cannot be instantiated directly. Instantiate 1 of the inherited classes instead, for a particular input type

Definition at line 63 of file TranslationOptionCollection.h.


Constructor & Destructor Documentation

Moses::TranslationOptionCollection::TranslationOptionCollection ( ttasksptr const &  ttask,
InputType const &  src 
) [protected]

constructor; since translation options are indexed by coverage span, the corresponding data structure is initialized here This fn should be called by inherited classe

Definition at line 54 of file TranslationOptionCollection.cpp.

References Moses::InputType::GetSize(), m_collection, and m_max_phrase_length.

Here is the call graph for this function:

Moses::TranslationOptionCollection::~TranslationOptionCollection (  )  [virtual]

destructor, clears out data structures

Definition at line 80 of file TranslationOptionCollection.cpp.

References m_inputPathQueue, and Moses::RemoveAllInColl().

Here is the call graph for this function:


Member Function Documentation

void Moses::TranslationOptionCollection::Add ( TranslationOption translationOption  )  [protected]

void Moses::TranslationOptionCollection::CacheLexReordering (  )  [protected]

Definition at line 615 of file TranslationOptionCollection.cpp.

References Moses::InputType::GetSize(), m_collection, m_source, Moses::LexicalReordering::SetCache(), and stop.

Referenced by Moses::TranslationOptionCollectionLattice::CreateTranslationOptions(), and CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CalcEstimatedScore (  )  [protected]

compute future score matrix in a dynamic programming fashion. This matrix used in search. Call this function once translation option collection has been filled with translation options

Definition at line 251 of file TranslationOptionCollection.cpp.

References Moses::TranslationOptionList::begin(), count, Moses::TranslationOptionList::end(), Moses::TranslationOption::GetFutureScore(), Moses::SquareMatrix::GetScore(), Moses::InputType::GetSize(), IFVERBOSE, Moses::SquareMatrix::InitTriangle(), m_collection, m_estimatedScores, m_source, Moses::SquareMatrix::SetScore(), Moses::TranslationOptionList::size(), and TRACE_ERR.

Referenced by Moses::TranslationOptionCollectionLattice::CreateTranslationOptions(), and CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CreateTranslationOptions (  )  [virtual]

Create all possible translations from the phrase tables.

Create all possible translations from the phrase tables for a particular input sentence. This implies applying all translation and generation steps. Also computes future cost matrix.

Reimplemented in Moses::TranslationOptionCollectionConfusionNet, Moses::TranslationOptionCollectionLattice, and Moses::TranslationOptionCollectionText.

Definition at line 334 of file TranslationOptionCollection.cpp.

References CacheLexReordering(), CalcEstimatedScore(), CreateTranslationOptionsForRange(), EvaluateWithSourceContext(), Moses::DecodeGraph::GetBackoff(), Moses::StaticData::GetDecodeGraphs(), Moses::InputType::GetSize(), Moses::StaticData::Instance(), m_collection, m_max_phrase_length, m_source, ProcessUnknownWord(), Prune(), Sort(), and VERBOSE.

Referenced by Moses::TranslationOptionCollectionConfusionNet::CreateTranslationOptions(), Moses::Manager::Decode(), and Moses::TrainingTask::Run().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual bool Moses::TranslationOptionCollection::CreateTranslationOptionsForRange ( const DecodeGraph decodeStepList,
size_t  startPosition,
size_t  endPosition,
bool  adhereTableLimit,
size_t  graphInd 
) [pure virtual]

Create translation options that exactly cover a specific input span.

Implemented in Moses::TranslationOptionCollectionConfusionNet, Moses::TranslationOptionCollectionLattice, and Moses::TranslationOptionCollectionText.

bool Moses::TranslationOptionCollection::CreateTranslationOptionsForRange ( const DecodeGraph decodeGraph,
size_t  startPos,
size_t  endPos,
bool  adhereTableLimit,
size_t  graphInd,
InputPath inputPath 
) [protected]

void Moses::TranslationOptionCollection::CreateXmlOptionsForRange ( size_t  startPosition,
size_t  endPosition 
) [virtual]

Create xml-based translation options for the specific input span.

Populates the current Collection with XML options exactly covering the range specified. Default implementation does nothing. called by CreateTranslationOptionsForRange()

Parameters:
sPos first position in input sentence
lastPos last position in input sentence

Reimplemented in Moses::TranslationOptionCollectionText.

Definition at line 571 of file TranslationOptionCollection.cpp.

Referenced by Moses::TranslationOptionCollectionConfusionNet::CreateTranslationOptionsForRangeLEGACY().

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::EvaluateTranslationOptionListWithSourceContext ( TranslationOptionList translationOptionList  )  [protected]

void Moses::TranslationOptionCollection::EvaluateWithSourceContext (  )  [protected]

Definition at line 490 of file TranslationOptionCollection.cpp.

References Moses::TranslationOptionList::begin(), Moses::TranslationOptionList::end(), EvaluateTranslationOptionListWithSourceContext(), Moses::InputType::GetSize(), m_collection, and m_source.

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual const SquareMatrix& Moses::TranslationOptionCollection::GetEstimatedScores (  )  const [inline, virtual]

returns future cost matrix for sentence

Definition at line 166 of file TranslationOptionCollection.h.

References m_estimatedScores.

Referenced by Moses::SearchCubePruning::CreateForwardTodos(), and Moses::SearchNormal::ExpandAllHypotheses().

Here is the caller graph for this function:

const InputPathList& Moses::TranslationOptionCollection::GetInputPaths (  )  const [inline]

Definition at line 176 of file TranslationOptionCollection.h.

References m_inputPathQueue.

const InputType& Moses::TranslationOptionCollection::GetSource (  )  const [inline]

input sentence/confusion network

Definition at line 132 of file TranslationOptionCollection.h.

References m_source.

Referenced by Moses::DecodeStepTranslation::ProcessLEGACY().

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::GetTargetPhraseCollectionBatch (  )  [protected]

TranslationOptionList const* Moses::TranslationOptionCollection::GetTranslationOptionList ( const Range coverage  )  const [inline]

list of trans opt for a particular span

Definition at line 172 of file TranslationOptionCollection.h.

References Moses::Range::GetEndPos(), Moses::Range::GetStartPos(), and GetTranslationOptionList().

Here is the call graph for this function:

TranslationOptionList const * Moses::TranslationOptionCollection::GetTranslationOptionList ( size_t  startPos,
size_t  endPos 
) const

Definition at line 641 of file TranslationOptionCollection.cpp.

References m_collection, NULL, and UTIL_THROW_IF2.

TranslationOptionList * Moses::TranslationOptionCollection::GetTranslationOptionList ( size_t  startPos,
size_t  endPos 
)

ttasksptr Moses::TranslationOptionCollection::GetTranslationTask (  )  const [inline]

Definition at line 180 of file TranslationOptionCollection.h.

References m_ttask.

Referenced by ProcessOneUnknownWord().

Here is the caller graph for this function:

const std::vector<const Phrase*>& Moses::TranslationOptionCollection::GetUnknownSources (  )  const [inline]

List of unknowns (OOVs).

Definition at line 137 of file TranslationOptionCollection.h.

References m_unksrcs.

Referenced by Moses::Manager::OutputUnknowns().

Here is the caller graph for this function:

bool Moses::TranslationOptionCollection::HasXmlOptionsOverlappingRange ( size_t  startPosition,
size_t  endPosition 
) const [virtual]

Check if this range has XML options.

Check if this range overlaps with any XML options. This doesn't need to be an exact match, only an overlap. by default, we don't support XML options. subclasses need to override this function. called by CreateTranslationOptionsForRange()

Parameters:
sPos first position in input sentence
lastPos last position in input sentence

Reimplemented in Moses::TranslationOptionCollectionText.

Definition at line 546 of file TranslationOptionCollection.cpp.

Referenced by Moses::TranslationOptionCollectionConfusionNet::CreateTranslationOptionsForRangeLEGACY().

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::ProcessOneUnknownWord ( const InputPath inputPath,
size_t  sourcePos,
size_t  length = 1,
const ScorePair inputScores = NULL 
) [protected, virtual]

special handling of ONE unknown words.

special handling of ONE unknown words. Either add temporarily add word to translation table, or drop the translation. This function should be called by the ProcessOneUnknownWord() in the inherited class At the moment, this unknown word handler is a bit of a hack, if copies over each factor from source to target word, or uses the 'UNK' factor. Ideally, this function should be in a class which can be expanded upon, for example, to create a morphologically aware handler.

Parameters:
sourceWord the unknown word
sourcePos 
length length covered by this word (may be > 1 for lattice input)
inputScores a set of scores associated with unknown word (input scores from latties/CNs)

Definition at line 174 of file TranslationOptionCollection.cpp.

References Add(), Moses::FactorCollection::AddFactor(), Moses::Phrase::AddWord(), Moses::ScoreComponentCollection::Assign(), EPSILON, Moses::TargetPhrase::EvaluateInIsolation(), StringPiece::find_first_of(), Moses::FloorScore(), Moses::PhraseDictionary::GetColl(), Moses::InputPath::GetPhrase(), Moses::TargetPhrase::GetScoreBreakdown(), Moses::Factor::GetString(), GetTranslationTask(), Moses::Phrase::GetWord(), Moses::FactorCollection::Instance(), Moses::UnknownWordPenaltyProducer::Instance(), m_unksrcs, NULL, range, Moses::TargetPhrase::SetAlignmentInfo(), Moses::TranslationOption::SetInputPath(), Moses::Word::SetIsOOV(), Moses::TransformScore(), and UNKNOWN_FACTOR.

Referenced by Moses::TranslationOptionCollectionLattice::CreateTranslationOptions(), Moses::TranslationOptionCollectionText::ProcessUnknownWord(), and Moses::TranslationOptionCollectionConfusionNet::ProcessUnknownWord().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual void Moses::TranslationOptionCollection::ProcessUnknownWord ( size_t  sourcePos  )  [protected, pure virtual]

void Moses::TranslationOptionCollection::ProcessUnknownWord (  )  [protected]

Force a creation of a translation option where there are none for a particular source position.

Force a creation of a translation option where there are none for a particular source position. ie. where a source word has not been translated, create a translation option by 1. not observing the table limits on phrase/generation tables 2. using the handler ProcessUnknownWord() Call this function once translation option collection has been filled with translation options

This function calls for unknown words is complicated by the fact it must handle different input types. The call stack is Base::ProcessUnknownWord() Inherited::ProcessUnknownWord(position) Base::ProcessOneUnknownWord()

Definition at line 130 of file TranslationOptionCollection.cpp.

References CreateTranslationOptionsForRange(), Moses::StaticData::GetDecodeGraphs(), Moses::InputType::GetSize(), GetTranslationOptionList(), Moses::StaticData::Instance(), m_source, m_ttask, and Moses::TranslationOptionList::size().

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::Prune (  )  [protected]

pruning: only keep the top n (m_maxNoTransOptPerCoverage) elements */

Definition at line 87 of file TranslationOptionCollection.cpp.

References Moses::InputType::GetSize(), m_collection, m_maxNoTransOptPerCoverage, m_source, m_translationOptionThreshold, Moses::TranslationOptionList::PruneByThreshold(), Moses::TranslationOptionList::SelectNBest(), Moses::TranslationOptionList::size(), and VERBOSE.

Referenced by Moses::TranslationOptionCollectionLattice::CreateTranslationOptions(), and CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::SetInputScore ( const InputPath inputPath,
PartialTranslOptColl oldPtoc 
) [protected]

void Moses::TranslationOptionCollection::Sort (  )  [protected]

sort all trans opt in each list for cube pruning */

Definition at line 520 of file TranslationOptionCollection.cpp.

References Moses::TranslationOptionList::begin(), Moses::TranslationOptionList::end(), Moses::InputType::GetSize(), m_collection, m_source, and sort().

Referenced by Moses::TranslationOptionCollectionLattice::CreateTranslationOptions(), and CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

Moses::TranslationOptionCollection::TO_STRING (  ) 

bool Moses::TranslationOptionCollection::ViolatesXmlOptionsConstraint ( size_t  startPosition,
size_t  endPosition,
TranslationOption transOpt 
) const [virtual]

Check if a subsumed XML option constraint is satisfied.

Check if an option conflicts with any constraint XML options. Okay, if XML option is substring in source and target. by default, we don't support XML options. subclasses need to override this function. called by CreateTranslationOptionsForRange()

Parameters:
sPos first position in input sentence
lastPos last position in input sentence

Reimplemented in Moses::TranslationOptionCollectionText.

Definition at line 559 of file TranslationOptionCollection.cpp.


Friends And Related Function Documentation

std::ostream& operator<< ( std::ostream &  out,
const TranslationOptionCollection coll 
) [friend]

Definition at line 598 of file TranslationOptionCollection.cpp.


Member Data Documentation

Definition at line 71 of file TranslationOptionCollection.h.

Referenced by CalcEstimatedScore(), and GetEstimatedScores().

Definition at line 72 of file TranslationOptionCollection.h.

Referenced by Prune().

Definition at line 73 of file TranslationOptionCollection.h.

Referenced by Prune().

std::vector<const Phrase*> Moses::TranslationOptionCollection::m_unksrcs [protected]

Definition at line 76 of file TranslationOptionCollection.h.

Referenced by GetUnknownSources(), and ProcessOneUnknownWord().

Definition at line 75 of file TranslationOptionCollection.h.


The documentation for this class was generated from the following files:

Generated on Thu Jul 6 00:33:56 2017 for Moses by  doxygen 1.5.9