Moses::LanguageModelImplementation Class Reference

Abstract base class which represent a language model on a contiguous phrase. More...

#include <Implementation.h>

Inheritance diagram for Moses::LanguageModelImplementation:

Inheritance graph
[legend]
Collaboration diagram for Moses::LanguageModelImplementation:

Collaboration graph
[legend]

List of all members.

Public Member Functions

virtual ~LanguageModelImplementation ()
void SetParameter (const std::string &key, const std::string &value)
virtual LMResult GetValueGivenState (const std::vector< const Word * > &contextFactor, FFState &state) const
virtual LMResult GetValueForgotState (const std::vector< const Word * > &contextFactor, FFState &outState) const =0
virtual void GetState (const std::vector< const Word * > &contextFactor, FFState &outState) const
 get State for a particular n-gram. We don't care what the score is.
virtual const FFState * GetNullContextState () const =0
virtual const FFState * GetBeginSentenceState () const =0
virtual FFState * NewState (const FFState *from=NULL) const =0
virtual void CalcScore (const Phrase &phrase, float &fullScore, float &ngramScore, size_t &oovCount) const
FFState * EvaluateWhenApplied (const Hypothesis &hypo, const FFState *ps, ScoreComponentCollection *out) const
 This interface should be implemented. Notes: When evaluating the value of this feature function, you should avoid calling hypo.GetPrevHypo(). If you need something from the "previous" hypothesis, you should store it in an FFState object which will be passed in as prev_state. If you don't do this, you will get in trouble.
FFState * EvaluateWhenApplied (const ChartHypothesis &cur_hypo, int featureID, ScoreComponentCollection *accumulator) const
void updateChartScore (float *prefixScore, float *finalScore, float score, size_t wordPos) const
size_t GetNGramOrder () const
 max n-gram order of LM
const Word & GetSentenceStartWord () const
 Contains factors which represents the beging and end words for this LM. Usually <s> and </s>.
const Word & GetSentenceEndWord () const
const FFState * EmptyHypothesisState (const InputType &) const
 return the state associated with the empty hypothesis for a given sentence

Protected Member Functions

 LanguageModelImplementation (const std::string &line)
 Usually <s> and </s>.

Protected Attributes

std::string m_filePath
size_t m_nGramOrder
Word m_sentenceStartWord
 max n-gram length contained in this LM
Word m_sentenceEndWord

Detailed Description

Abstract base class which represent a language model on a contiguous phrase.

Definition at line 50 of file Implementation.h.

Constructor & Destructor Documentation

Moses::LanguageModelImplementation::LanguageModelImplementation ( const std::string &  line  )  [protected]

Usually <s> and </s>.

Contains factors which represents the beging and end words for this LM.

Definition at line 43 of file Implementation.cpp.

virtual Moses::LanguageModelImplementation::~LanguageModelImplementation (  )  [inline, virtual]

Definition at line 66 of file Implementation.h.

Member Function Documentation

void Moses::LanguageModelImplementation::CalcScore ( const Phrase &  phrase,
float &  fullScore,
float &  ngramScore,
size_t &  oovCount 
) const [virtual]

Reimplemented in Moses::LanguageModelIRST.

Definition at line 89 of file Implementation.cpp.

References GetBeginSentenceState(), GetNGramOrder(), GetNullContextState(), GetSentenceStartWord(), Moses::Phrase::GetSize(), GetValueGivenState(), Moses::Phrase::GetWord(), Moses::Word::IsNonTerminal(), NewState(), Moses::LMResult::score, state, Moses::LMResult::unknown, UTIL_THROW2, and UTIL_THROW_IF2.

Here is the call graph for this function:

const FFState* Moses::LanguageModelImplementation::EmptyHypothesisState ( const InputType &  input  )  const [inline, virtual]

return the state associated with the empty hypothesis for a given sentence

Implements Moses::LanguageModel.

Reimplemented in Moses::LanguageModelIRST.

Definition at line 110 of file Implementation.h.

References GetBeginSentenceState(), and NewState().

Here is the call graph for this function:

FFState * Moses::LanguageModelImplementation::EvaluateWhenApplied ( const ChartHypothesis &  cur_hypo,
int  featureID,
ScoreComponentCollection *  accumulator 
) const [virtual]

Implements Moses::StatefulFeatureFunction.

Definition at line 225 of file Implementation.cpp.

References Moses::TargetPhrase::GetAlignNonTerm(), GetBeginSentenceState(), Moses::ChartHypothesis::GetCurrTargetPhrase(), Moses::ChartHypothesis::GetFFState(), GetNGramOrder(), Moses::AlignmentInfo::GetNonTermIndexMap(), GetNullContextState(), Moses::LanguageModelChartState::GetNumTargetTerminals(), Moses::LanguageModelChartState::GetPrefix(), Moses::LanguageModelChartState::GetPrefixScore(), Moses::ChartHypothesis::GetPrevHypo(), Moses::LanguageModelChartState::GetRightContext(), Moses::ChartTranslationOption::GetScores(), Moses::ScoreComponentCollection::GetScoresForProducer(), GetSentenceStartWord(), Moses::Phrase::GetSize(), Moses::LanguageModelChartState::GetSuffix(), Moses::ChartHypothesis::GetTranslationOption(), GetValueGivenState(), Moses::Phrase::GetWord(), Moses::Word::IsNonTerminal(), NewState(), Moses::LanguageModel::OOVFeatureEnabled(), Moses::ScoreComponentCollection::PlusEquals(), Moses::LanguageModelChartState::Set(), updateChartScore(), and UTIL_THROW_IF2.

Here is the call graph for this function:

FFState * Moses::LanguageModelImplementation::EvaluateWhenApplied ( const Hypothesis &  cur_hypo,
const FFState *  prev_state,
ScoreComponentCollection *  accumulator 
) const [virtual]

This interface should be implemented. Notes: When evaluating the value of this feature function, you should avoid calling hypo.GetPrevHypo(). If you need something from the "previous" hypothesis, you should store it in an FFState object which will be passed in as prev_state. If you don't do this, you will get in trouble.

Implements Moses::StatefulFeatureFunction.

Reimplemented in Moses::LanguageModelIRST.

Definition at line 137 of file Implementation.cpp.

References Moses::Hypothesis::GetCurrTargetLength(), Moses::Hypothesis::GetCurrTargetWordsRange(), Moses::Range::GetEndPos(), Moses::Hypothesis::GetManager(), GetNGramOrder(), GetSentenceEndWord(), GetSentenceStartWord(), Moses::Manager::GetSentenceStats(), Moses::Hypothesis::GetSize(), Moses::Range::GetStartPos(), GetState(), GetValueForgotState(), GetValueGivenState(), Moses::Hypothesis::GetWord(), IFVERBOSE, index, Moses::Hypothesis::IsSourceCompleted(), NewState(), NULL, Moses::LanguageModel::OOVFeatureEnabled(), Moses::ScoreComponentCollection::PlusEquals(), Moses::LMResult::score, Moses::SentenceStats::StartTimeCalcLM(), and Moses::SentenceStats::StopTimeCalcLM().

Here is the call graph for this function:

virtual const FFState* Moses::LanguageModelImplementation::GetBeginSentenceState (  )  const [pure virtual]

Implemented in Moses::LanguageModelSingleFactor.

Referenced by CalcScore(), EmptyHypothesisState(), and EvaluateWhenApplied().

Here is the caller graph for this function:

size_t Moses::LanguageModelImplementation::GetNGramOrder (  )  const [inline]

max n-gram order of LM

Definition at line 98 of file Implementation.h.

References m_nGramOrder.

Referenced by CalcScore(), EvaluateWhenApplied(), and updateChartScore().

Here is the caller graph for this function:

virtual const FFState* Moses::LanguageModelImplementation::GetNullContextState (  )  const [pure virtual]

Implemented in Moses::LanguageModelSingleFactor.

Referenced by CalcScore(), and EvaluateWhenApplied().

Here is the caller graph for this function:

const Word& Moses::LanguageModelImplementation::GetSentenceEndWord (  )  const [inline]

Definition at line 106 of file Implementation.h.

References m_sentenceEndWord.

Referenced by EvaluateWhenApplied().

Here is the caller graph for this function:

const Word& Moses::LanguageModelImplementation::GetSentenceStartWord (  )  const [inline]

Contains factors which represents the beging and end words for this LM. Usually <s> and </s>.

Definition at line 103 of file Implementation.h.

References m_sentenceStartWord.

Referenced by CalcScore(), and EvaluateWhenApplied().

Here is the caller graph for this function:

void Moses::LanguageModelImplementation::GetState ( const std::vector< const Word * > &  contextFactor,
FFState &  outState 
) const [virtual]

get State for a particular n-gram. We don't care what the score is.

Definition at line 81 of file Implementation.cpp.

References GetValueForgotState().

Referenced by EvaluateWhenApplied().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual LMResult Moses::LanguageModelImplementation::GetValueForgotState ( const std::vector< const Word * > &  contextFactor,
FFState &  outState 
) const [pure virtual]

Implemented in Moses::LanguageModelSingleFactor.

Referenced by EvaluateWhenApplied(), GetState(), and GetValueGivenState().

Here is the caller graph for this function:

LMResult Moses::LanguageModelImplementation::GetValueGivenState ( const std::vector< const Word * > &  contextFactor,
FFState &  state 
) const [virtual]

Definition at line 74 of file Implementation.cpp.

References GetValueForgotState().

Referenced by CalcScore(), and EvaluateWhenApplied().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual FFState* Moses::LanguageModelImplementation::NewState ( const FFState *  from = NULL  )  const [pure virtual]

Implemented in Moses::LanguageModelSingleFactor.

Referenced by CalcScore(), EmptyHypothesisState(), and EvaluateWhenApplied().

Here is the caller graph for this function:

void Moses::LanguageModelImplementation::SetParameter ( const std::string &  key,
const std::string &  value 
) [virtual]

Reimplemented from Moses::LanguageModel.

Reimplemented in Moses::LanguageModelIRST, Moses::OxLM< Model >, and Moses::LanguageModelSingleFactor.

Definition at line 49 of file Implementation.cpp.

References m_filePath, and m_nGramOrder.

void Moses::LanguageModelImplementation::updateChartScore ( float *  prefixScore,
float *  finalScore,
float  score,
size_t  wordPos 
) const

Definition at line 352 of file Implementation.cpp.

References GetNGramOrder().

Referenced by EvaluateWhenApplied().

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

std::string Moses::LanguageModelImplementation::m_filePath [protected]

Definition at line 57 of file Implementation.h.

Referenced by Moses::OxLM< Model >::CleanUpAfterSentenceProcessing(), Moses::OxLM< Model >::InitializeForInput(), Moses::LanguageModelIRST::LanguageModelIRST(), Moses::LanguageModelSRI::Load(), Moses::LanguageModelRandLM::Load(), Moses::OxLM< Model >::Load(), Moses::NeuralLMWrapper::Load(), Moses::LanguageModelMaxEntSRI::Load(), Moses::LanguageModelIRST::Load(), SetParameter(), and Moses::OxLM< Model >::~OxLM().

size_t Moses::LanguageModelImplementation::m_nGramOrder [protected]

Definition at line 58 of file Implementation.h.

Referenced by Moses::ExampleLM::ExampleLM(), GetNGramOrder(), Moses::LanguageModelRemote::GetValue(), Moses::LanguageModelSRI::Load(), Moses::LanguageModelRemote::Load(), Moses::LanguageModelRandLM::Load(), Moses::OxLM< Model >::Load(), Moses::NeuralLMWrapper::Load(), Moses::LanguageModelMaxEntSRI::Load(), Moses::LanguageModelIRST::Load(), Moses::LanguageModelIRST::SetParameter(), and SetParameter().

Word Moses::LanguageModelImplementation::m_sentenceEndWord [protected]

Definition at line 59 of file Implementation.h.

Referenced by Moses::LanguageModelSRI::CreateFactors(), Moses::LanguageModelRandLM::CreateFactors(), Moses::LanguageModelMaxEntSRI::CreateFactors(), Moses::LanguageModelIRST::CreateFactors(), Moses::ExampleLM::ExampleLM(), GetSentenceEndWord(), and Moses::NeuralLMWrapper::Load().

Word Moses::LanguageModelImplementation::m_sentenceStartWord [protected]

max n-gram length contained in this LM

Definition at line 59 of file Implementation.h.

Referenced by Moses::LanguageModelSRI::CreateFactors(), Moses::LanguageModelRandLM::CreateFactors(), Moses::LanguageModelMaxEntSRI::CreateFactors(), Moses::LanguageModelIRST::CreateFactors(), Moses::ExampleLM::ExampleLM(), GetSentenceStartWord(), and Moses::NeuralLMWrapper::Load().

The documentation for this class was generated from the following files:
Generated on Thu Jul 6 00:33:21 2017 for Moses by  doxygen 1.5.9