



\section{
\label{training-reference.tex}
Reference: All Training Parameters}

\begin{compactitem}
\item {\tt -{}-root-dir} -{}- root directory, where output files are stored

\item {\tt -{}-corpus} -{}- corpus file name (full pathname), excluding extension

\item {\tt -{}-e} -{}- extension of the English corpus file

\item {\tt -{}-f} -{}- extension of the foreign corpus file

\item {\tt -{}-lm} -{}- language model: $<$factor$>$:$<$order$>$:$<$filename$>$ (option can be repeated)

\item {\tt -{}-first-step} -{}- first step in the training process (default 1)

\item {\tt -{}-last-step} -{}- last step in the training process (default 7)

\item {\tt -{}-parts} -{}- break up corpus in smaller parts before GIZA++ training

\item {\tt -{}-corpus-dir} -{}- corpus directory (default {\tt \$ROOT/corpus})

\item {\tt -{}-lexical-dir} -{}- lexical translation probability directory (default {\tt \$ROOT/model})

\item {\tt -{}-model-dir} -{}- model directory (default {\tt \$ROOT/model})

\item {\tt -{}-extract-file} -{}- extraction file (default {\tt \$ROOT/model/extract})

\item {\tt -{}-giza-f2e} -{}- GIZA++ directory (default {\tt \$ROOT/giza.\$F-\$E})

\item {\tt -{}-giza-e2f} -{}- inverse GIZA++ directory (default {\tt \$ROOT/giza.\$E-\$F})

\item {\tt -{}-alignment} -{}- heuristic used for word alignment: {\tt intersect}, {\tt union}, {\tt grow}, {\tt grow-final}, {\tt grow-diag}, {\tt grow-diag-final} (default), {\tt grow-diag-final-and}, {\tt srctotgt}, {\tt tgttosrc}

\item {\tt -{}-max-phrase-length} -{}- maximum length of phrases entered into phrase table (default 7)

\item {\tt -{}-giza-option} -{}- additional options for GIZA++ training

\item {\tt -{}-verbose} -{}- prints additional word alignment information

\item {\tt -{}-no-lexical-weighting} -{}- only use conditional probabilities for the phrase table, not lexical weighting

\item {\tt -{}-parts} -{}- prepare data for GIZA++ by running {\tt snt2cooc} in parts

\item {\tt -{}-direction} -{}- run training step 2 only in direction 1 or 2 (for parallelization)

\item {\tt -{}-reordering} -{}- specifies which reordering models to train using a comma-separated list of config-strings, see FactoredTraining.BuildReorderingModel (Section \ref{training7.tex}). (default distance)

\item {\tt -{}-reordering-smooth} -{}- specifies the smoothing constant to be used for training lexicalized reordering models. If the letter "u" follows the constant, smoothing is based on actual counts. (default 0.5)

\item {\tt -{}-alignment-factors} -{}- 

\item {\tt -{}-translation-factors} -{}- 

\item {\tt -{}-reordering-factors} -{}- 

\item {\tt -{}-generation-factors} -{}- 

\item {\tt -{}-decoding-steps} -{}- 

\end{compactitem}

\subsection{
Basic Options}


A number of parameters are required to point the training script 
to the correct training data. We will describe them in this section.
Other options allow for partial training runs and alternative
settings.



As mentioned before, you want to create a special directory for
training.
The path to that directory has to be specified with the parameter
{\tt -{}-root-dir}. 



The root directory has to contain a sub directory (called {\tt corpus})
that contains the training data. The training data is a parallel
corpus, stored in two files, one for the English
sentences, one for the foreign sentences. The corpus has to be
sentence-aligned, meaning that the 1624th line in the English file
is the translation of the 1624th line in the foreign file.



Typically, the data is lowercased, no empty lines are allowed,
and having multiple spaces between words may cause problems.
Also, sentence length is limited to 100 words per sentence.
The sentence length ratio for a sentence pair can be at most
9 (i.e, having a 10-word sentence aligned to a 1-word
sentence is disallowed). These restrictions on sentence length
are caused by GIZA++ and may be changed (see below).



The two corpus files have a common file stem (say, {\tt euro})
and extensions indicating the language (say, {\tt en} and {\tt de}).
The file stem ({\tt -{}-corpus-file}), and the language extensions
({\tt -{}-e} and {\tt -{}-f}) have to be specified to the training script.



In summary, the training script may be invoked as follows:






\begin{SaveVerbatim}{myverb}
train-model.perl --root-dir . --f de --e en --corpus corpus/euro >& LOG
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

After training, typically the following files can be found in the root directory (note the time stamps that tell you something about how much time was spent on each step took for this data):






\begin{SaveVerbatim}{myverb}
> ls -lh *
-rw-rw-r--    1 koehn    user         110K Jul 13 21:49 LOG

corpus:
total 399M
-rw-rw-r--    1 koehn    user         104M Jul 12 19:58 de-en-int-train.snt
-rw-rw-r--    1 koehn    user         4.2M Jul 12 19:56 de.vcb
-rw-rw-r--    1 koehn    user         3.2M Jul 12 19:42 de.vcb.classes
-rw-rw-r--    1 koehn    user         2.6M Jul 12 19:42 de.vcb.classes.cats
-rw-rw-r--    1 koehn    user         104M Jul 12 19:59 en-de-int-train.snt
-rw-rw-r--    1 koehn    user         1.1M Jul 12 19:56 en.vcb
-rw-rw-r--    1 koehn    user         793K Jul 12 19:56 en.vcb.classes
-rw-rw-r--    1 koehn    user         614K Jul 12 19:56 en.vcb.classes.cats
-rw-rw-r--    1 koehn    user          94M Jul 12 18:08 euro.de
-rw-rw-r--    1 koehn    user          84M Jul 12 18:08 euro.en

giza.de-en:
total 422M
-rw-rw-r--    1 koehn    user         107M Jul 13 03:57 de-en.A3.final.gz
-rw-rw-r--    1 koehn    user         314M Jul 12 20:11 de-en.cooc
-rw-rw-r--    1 koehn    user         2.0K Jul 12 20:11 de-en.gizacfg

giza.en-de:
total 421M
-rw-rw-r--    1 koehn    user         107M Jul 13 11:03 en-de.A3.final.gz
-rw-rw-r--    1 koehn    user         313M Jul 13 04:07 en-de.cooc
-rw-rw-r--    1 koehn    user         2.0K Jul 13 04:07 en-de.gizacfg

model:
total 2.1G
-rw-rw-r--    1 koehn    user          94M Jul 13 19:59 aligned.de
-rw-rw-r--    1 koehn    user          84M Jul 13 19:59 aligned.en
-rw-rw-r--    1 koehn    user          90M Jul 13 19:59 aligned.grow-diag-final
-rw-rw-r--    1 koehn    user         214M Jul 13 20:33 extract.gz
-rw-rw-r--    1 koehn    user         212M Jul 13 20:35 extract.inv.gz
-rw-rw-r--    1 koehn    user          78M Jul 13 20:23 lex.f2n
-rw-rw-r--    1 koehn    user          78M Jul 13 20:23 lex.n2f
-rw-rw-r--    1 koehn    user          862 Jul 13 21:49 pharaoh.ini
-rw-rw-r--    1 koehn    user         1.2G Jul 13 21:49 phrase-table
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

Summary

\begin{compactitem}
\item {\tt -{}-root-dir} -{}- root directory, where output files are stored

\item {\tt -{}-corpus} -{}- corpus, expected in {\tt \$ROOT/corpus}

\item {\tt -{}-e} -{}- extension of the English corpus file

\item {\tt -{}-f} -{}- extension of the foreign corpus file

\item {\tt -{}-lm} -{}- language model file

\end{compactitem}

\subsection{
Factored Translation Model Settings}


More on factored translation models in the Overviev (Section \ref{training-intro.tex}).



Summary

\begin{compactitem}
\item {\tt -{}-alignment-factors} -{}- 

\item {\tt -{}-translation-factors} -{}- 

\item {\tt -{}-reordering-factors} -{}- 

\item {\tt -{}-generation-factors} -{}- 

\item {\tt -{}-decoding-steps} -{}- 

\end{compactitem}

\subsection{
Lexicalized Reordering Model}


More on lexicalized reording on the description of Training step 7: build reordering model (Section \ref{training7.tex}).



Summary

\begin{compactitem}
\item {\tt -{}-reordering} -{}- 

\item {\tt -{}-reordering-smooth} -{}- 

\end{compactitem}

\subsection{
Partial Training}


You may have better ideas how to do word alignment, 
extract phrases or score phrases. Since the training is modular,
you can start training at any of the seven training steps 
{\tt -{}-first-step} and 
end it at any subsequent step {\tt -{}-last-step}.



Again, the nine training steps are:

\begin{compactenum}\item Prepare data

\item Run GIZA++

\item Align words

\item Get lexical translation table

\item Extract phrases

\item Score phrases

\item Build reordering model

\item Build generation models 

\item Create configuration file

\end{compactenum}
For instance, if you may have your own method to generate a
word alignment, you want to skip these training steps and 
start with lexical translation table generation, you may specify this by






\begin{SaveVerbatim}{myverb}
train-model.perl [...] --first-step 4
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

Summary

\begin{compactitem}
\item {\tt -{}-first-step} -{}- first step in the training process (default 1)

\item {\tt -{}-last-step} -{}- last step in the training process (default 7)

\end{compactitem}

\subsection{
File Locations}


A number of parameters allow you to break out of the rigid file
name conventions of the training script. A typical use for this
is that you want to try alternative training runs, but there is no
need to repeat all the training steps.



For instance, you may want to try an alternative alignment heuristic.
There is no need to rerun GIZA++. You could copy the 
necessary files from the {\tt corpus} and the {\tt giza.*}
directories into a new root directory, but this takes up a lot
of additional disk space and makes the file organization
unnecessarily complicated.



Since you only need a new model directory, you can specify 
this with the parameter {\tt -{}-model-dir}, and stay within the
precious root directory structure:






\begin{SaveVerbatim}{myverb}
train-model.perl [...] --first-step 3 --alignment union --model-dir model-union
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

The other parameters for file and directory names fullfill similar 
purposes.



Summary

\begin{compactitem}
\item {\tt -{}-corpus-dir} -{}- corpus directory (default {\tt \$ROOT/corpus})

\item {\tt -{}-lexical-dir} -{}- lexical translation probability directory (default {\tt \$ROOT/model})

\item {\tt -{}-model-dir} -{}- model directory (default {\tt \$ROOT/model})

\item {\tt -{}-extract-file} -{}- extraction file (default {\tt \$ROOT/model/extract})

\item {\tt -{}-giza-f2e} -{}- GIZA++ directory (default {\tt \$ROOT/giza.\\$F-\\$E}$\}$

\item {\tt -{}-giza-e2f} -{}- inverse GIZA++ directory (default {\tt \$ROOT/giza.\\$E-\\$F})

\end{compactitem}

\subsection{
Alignment Heuristic}


A number of different word alignment heuristics are implemented, and can be 
specified with the parameter {\tt -{}-alignment}.
The options are:






\begin{compactitem}
\item {\tt intersect} -{}- the intersection of the two GIZA++ alignments is taken. This usually creates a lot of extracted phrases, since the unaligned words create a lot of freedom to align phrases.

\item {\tt union} -{}- the union of the two GIZA++ alignments is  taken

\item {\tt grow-diag-final} -{}- the default heuristic 

\item {\tt grow-diag} -{}- same as above, but without a call to function {\tt FINAL()} (see background to word alignment).

\item {\tt grow} -{}- same as above, but with a different definition of {\em neighboring}. Now diagonally adjacent alignment points are excluded.

\item {\tt grow} -{}- no diagonal neighbors, but with {\tt FINAL()}

\end{compactitem}

Different heuristic may show better performance for a specific
language pair or corpus, so some experimentation may be useful.



Summary

\begin{compactitem}
\item {\tt -{}-alignment} -{}- heuristic used for word alignment: intersect, union, grow, grow-final, grow-diag, grow-diag-final (default)

\end{compactitem}

\subsection{
Maximum Phrase Length}


The maximum length of phrases is limited to 7 words. The maximum phrase 
length impacts the size of the phrase translation table, so shorter 
limits may be desirable, if phrase table size is an issue. 
Previous experiments have shown that performance increases only slightly
when including phrases of more that 3 words.



Summary

\begin{compactitem}
\item {\tt -{}-max-phrase-length} -{}- maximum length of phrases entered into phrase table (default 7)

\end{compactitem}

\subsection{
GIZA++ Options}


GIZA++ takes a lot of parameters to specify the behavior of the training
process and limits on sentence length, etc. Please refer to 
the corresponding documentation for details on this.



Parameters can be passed on to GIZA++ with the switch {\tt -{}-giza-option}.



For instance, if you want to the change the number of iterations
for the different IBM models to 4 iterations of Model 1, 
0 iterations of Model 2, 4 iterations of the HMM Model,
0 iterations of Model 3, and 3 iterations of Model 4,
you can specify this by






\begin{SaveVerbatim}{myverb}
train-model.perl [...] --giza-option m1=4,m2=0,mh=4,m3=0,m4=3
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

Summary

\begin{compactitem}
\item {\tt -{}-giza-option} -{}- additional options for GIZA++ training

\end{compactitem}

\subsection{
Dealing with large training corpora}


Training on large training corpora may become a problem for the GIZA++ word alignment tool. Since it stores the word translation table in memory, the size of this table may become too large for the available RAM of the machine. For instance, the data sets for the NIST Arabic-English and Chinese-English competitions require more than 4 GB of RAM, which is a problem for current 32-bit machines.



This problem can be remedied to some degree by a more efficient data structure in GIZA++, which requires the run of {\tt snt2cooc} in advance on the corpus in parts and the merging on the resulting output. All you need to know is that running the training script with the option {\tt -{}-parts n}, e.g. {\tt -{}-parts 3} may allow you to train on a corpus that was too large for a regular run.



Somewhat related to this problem caused by large training corpora is the problem of the large run time of GIZA++. It is possible to run the two GIZA++ separately on two machines with the switch {\tt -{}-direction}. When running one of the runs on one machine with {\tt -{}-direction 1} and the other run on a different machine or CPU with {\tt -{}-direction 2}, the processing time for training step 2 can be cut in half.



Summary

\begin{compactitem}
\item {\tt -{}-parts} -{}- prepare data for GIZA++ by running {\tt snt2cooc} in parts

\item {\tt -{}-direction} -{}- run training step 2 only in direction 1 or 2 (for parallelization)

\end{compactitem}






\begin{center}\textit{\small Subsection last  modified on July 28, 2013, at 08:45 AM}\end{center}
