



\section{
\label{training7.tex}
Training Step 7: Build reordering model}


By default, only a distance-based reordering model is included in final configuration. This model gives a cost linear to the reordering distance. For instance, skipping over two words costs twice as much as skipping over one word.



However, additional conditional reordering models, so called lexicalized reordering models, may be build. There are three types of lexicalized reordering models in Moses that are based on Koehn et al. (2005)\footnote{\sf http://homepages.inf.ed.ac.uk/pkoehn/publications/iwslt05-report.pdf} and Galley and Manning (2008)\footnote{\sf http://www.aclweb.org/anthology/D/D08/D08-1089.pdf}. The Koehn at al. model determines the orientation of two phrases based on word alignments at training time, and based on phrase alignments at decoding time. The other two models are based on Galley and Manning. The phrase-based model uses phrases both at training and decoding time, and the hierarchical model allows combinations of several phrases for determining the orientation.



The lexicalized reordering models are specified by a configuration string, containing five parts, that account for different aspects:

\begin{compactitem}
\item {\bf Modeltype} - the type of model used (see above)
\begin{compactitem}
\item {\tt wbe} - word-based extraction (but phrase-based at decoding). This is the original model in Moses. {\bf DEFAULT} 

\item {\tt phrase} - phrase-based model

\item {\tt hier} - hierarchical model

\end{compactitem}

\item {\bf Orientation} - Which classes of orientations that are used in the model
\begin{compactitem}
\item {\tt mslr} - Considers four different orientations: {\tt monotone}, {\tt swap}, {\tt discontinuous-left}, {\tt discontinuous-right}

\item {\tt msd} - Considers three different orientations: {\tt monotone}, {\tt swap}, {\tt discontinuous} (the two discontinuous classes of the mslr model are merged into one class)

\item {\tt monotonicity} - Considers two different orientations: {\tt monotone} or {\tt non-monotone} ({\tt swap} and {\tt discontinuous} of the msd model are merged into the {\tt non-monotone} class)

\item {\tt leftright} - Considers two different orientations: {\tt left} or {\tt right} (the four classes in the {\tt mslr} model are merged into two classes, {\tt swap} and {\tt discontinuous-left} into {\tt left} and {\tt monotone} and {\tt discontinuous-right} into {\tt right})

\end{compactitem}

\item {\bf Directionality} - Determines if the orientation should be modeled based on the previous or next phrase, or both.
\begin{compactitem}
\item {\tt backward} - determine orientation with respect to previous phrase {\bf DEFAULT}

\item {\tt forward} - determine orientation with respect to following phrase

\item {\tt bidirectional} - use both backward and forward models 

\end{compactitem}

\item {\bf language} - decides which language to base the model on
\begin{compactitem}
\item {\tt fe} - conditioned on both the source and target languages

\item {\tt f} - conditioned on the source language only

\end{compactitem}

\item {\bf collapsing} - determines how to treat the scores
\begin{compactitem}
\item {\tt allff} - treat the scores as individual feature functions {\bf DEFAULT}

\item {\tt collapseff} - collapse all scores in one direction into one feature function 

\end{compactitem}

\end{compactitem}

any possible configuration of these five factors is allowed. It is always necessary to specify orientation and language. The other three factors use the default values indicated above if they are not specified. Some examples of possible models are:

\begin{compactitem}
\item {\tt msd-bidirectional-fe} (this model is commonly used, for instance it is the model used in the WMT baselines\footnote{\sf http://www.statmt.org/wmt11/baseline.html})

\item {\tt wbe-msd-bidirectional-fe-allff} same model as above

\item {\tt mslr-f}

\item {\tt wbe-backward-mslr-f-allff} same model as above

\item {\tt phrase-msd-bidirectional-fe}

\item {\tt hier-mslr-bidirectional-fe}

\item {\tt hier-leftright-forward-f-collapseff}

\end{compactitem}

and of course {\tt distance}.



Which reordering model(s) that are used (and built during the training process, if necessary) can be set with the switch {\tt -reordering}, e.g.:

\begin{SaveVerbatim}{myverb}
-reordering distance
-reordering msd-bidirectional-fe
-reordering msd-bidirectional-fe,hier-mslr-bidirectional-fe
-reordering distance,msd-bidirectional-fe,hier-mslr-bidirectional-fe
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

Note that the distance model is always included, so there is no need to specify it.



The number of features that are created with a lexical reordering model depends on the type of the model. If the flag {\tt allff} is used, a {\tt msd} model has three features, one each for the probability that the phrase is translated monotone, swapped, or discontinuous, a {\tt mslr} model has four features and a {\tt monotonicity} or {\tt leftright} model has two features. If a {\tt bidirectional} model is used, then the number of features doubles - one for each direction. If {\tt collapseff} are used there is one feature for each direction, regardless of which orientation types that are used.



There are also a number of other flags that can be given to {\tt train-model.perl} that concerns the reordering models:

\begin{compactitem}
\item {\tt -{}-reordering-smooth} - specifies the smoothing constant to be used for training lexicalized reordering models. If the letter {\tt u} follows the constant, smoothing is based on actual counts. (default 0.5) 

\item {\tt -{}-max-lexical-reordering} - if this flag is used, the extract file will contain information for the mslr orientations for all three model types, {\tt wbe}, {\tt phrase} and {\tt hier}. Otherwise the extract file will contain the minimum information that is needed based on which reordering model config strings that are given. 

\end{compactitem}






\begin{center}\textit{\small Subsection last  modified on July 28, 2013, at 03:55 AM}\end{center}
