



\section{
\label{code-guide.tex}
Code Guide}


\subsection{
Github, branching, and merging}


If you want to code with Moses, you should create your own repository in one of a number of ways. 



The preference is that you fork the repository if you're doing long-term research. If you fixed a bug, please commit it yourself, or create a pull request. 






\begin{compactitem}
\item {\bf Clone the moses github repository} to your hard disk and work with it:

\end{compactitem}
\begin{SaveVerbatim}{myverb}
git clone https://github.com/moses-smt/mosesdecoder.git mymoses
cd mymoses
edit files ....
git commit -am "Check in"
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

You don't need a github login or permission to do this. All changes are stored on your own hard disk






\begin{compactitem}
\item {\bf Clone AND branch the repository}:

\end{compactitem}
\begin{SaveVerbatim}{myverb}
git clone https://github.com/moses-smt/mosesdecoder.git mymoses
cd mymoses
git checkout -b mybranch
edit files ....
git commit -am "Check in"
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

You still don't need a github login or permission to do this. 






\begin{compactitem}
\item {\bf Clone and branch AND push to github}:

\end{compactitem}
\begin{SaveVerbatim}{myverb}
git clone https://github.com/moses-smt/mosesdecoder.git mymoses
cd mymoses
git checkout -b mybranch
edit files ....
git commit -am "Check in"
git push origin mybranch
edit files ....
git commit -am "Check in again"
git push
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

You need a github account. And you have to ask one of the Moses administrators to add you as a committer to the Moses repository.



NB. To delete a LOCAL branch:

\begin{SaveVerbatim}{myverb}
git branch -D old-branch
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

To delete a branch on the github server:

\begin{SaveVerbatim}{myverb}
git push origin --delete old-branch
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}




\begin{compactitem}
\item {\bf Fork the repository}. You need a github account. You don't need permission from the Moses administrators. Log into github.com on their webiste, and go to the Moses page:

\end{compactitem}
\begin{SaveVerbatim}{myverb}
https://github.com/moses-smt/mosesdecoder
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

Press the {\bf Fork} button. This creates a new repository only you have write access to. Clone that repository and do whatever you want. Eg.

\begin{SaveVerbatim}{myverb}
git clone https://github.com/hieuhoang/mosesdecoder.git
cd hieuhoang
edit files ....
git commit -am "Check in again"
git push
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}
\begin{compactitem}
\item {\bf Clone and check into master}

\end{compactitem}
\begin{SaveVerbatim}{myverb}
git clone https://github.com/moses-smt/mosesdecoder.git mymoses
cd mymoses
edit files ....
git commit -am "Check in"
git push
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

You need a github account and write permission to the Moses repository. 






\begin{compactitem}
\item {\bf Create pull request}. Fork a repository and read the instructions here:

\end{compactitem}
\begin{SaveVerbatim}{myverb}
https://help.github.com/articles/using-pull-requests
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

\subsubsection*{
Working with multiple branches}


Assuming you've done {\bf Fork the repository}, you can merge the latest changes from the main Moses repository with this command:

\begin{SaveVerbatim}{myverb}
git pull https://github.com/moses-smt/mosesdecoder.git
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

In your own repository, you can create branches and switch between them using

\begin{SaveVerbatim}{myverb}
git checkout -b new-branch
edit files...
git commit -am "check in"

git checkout master
edit files...
...
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

To get the latest changes from the main Moses repository to your branch, on your fork:

\begin{SaveVerbatim}{myverb}
git checkout master
git pull https://github.com/moses-smt/mosesdecoder.git
git checkout new-branch
git merge master
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

\subsubsection*{
Regression test}


If you've changed any of the C++ code and intend to check into the main Moses repository, please run the {\bf regression test} to make sure you haven't broken anything:

\begin{SaveVerbatim}{myverb}
git submodule init
git submodule update
./bjam with-irstlm=... --with-srilm=... {\bf -a --with-regtest} >& reg.out &
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

Check the output for any failures:

\begin{SaveVerbatim}{myverb}
grep FAIL reg.out
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

\subsubsection*{
Contact the administrators}


Contact Hieu Hoang or Barry Haddow, or any of the other administrators you might know, if you need help or permission to the github repository.   



\subsection{
The code}


This section gives a overview of the code. All the source code is commented for Doxygen, so you can browse a current snapshot of the source documentation\footnote{\sf http://www.statmt.org/moses/html/hierarchy.html}. Moses is implemented using object-oriented principles, and you can get a good idea of its class organization from this documentation



The source code is in the following directories

\begin{compactitem}
\item {\tt moses/util} contains some shared utilities, such as code for reading and parsing files, and for hash tables. This was originally part of KenLM.

\item {\tt moses/lm} contains KenLM, Moses default language model.

\item {\tt moses/src} contains the code for the decoder

\item {\tt moses/src/LM} contains the language model wrappers used by the decoder

\item Other subdirectories of {\tt moses/src} contain some more specialised parts of the decoder, such as alternative chart decoding algorithms.

\item {\tt moses-cmd/src} contains code relevant to the command line version of the phrase-based decoder

\item {\tt moses-cmd/src} contains code relevant to the command line version of the chart-based decoder

\item {\tt mert} contains the code for the Moses mert implementation, originally described here\footnote{\sf http://homepages.inf.ed.ac.uk/bhaddow/prague-mert.pdf}

\end{compactitem}

In the following, we provide a short walk-through of the decoder.



\subsection{
Quick Start}

\begin{compactitem}
\item The main function: {\tt moses-cmd/src/Main.cpp} 

\item Initialize the decoder 
\begin{compactitem}
\item {\tt moses/src/Parameter.cpp} specifies parameters 

\item {\tt moses/src/StaticData.cpp} contains globals, loads tables 

\end{compactitem}

\item Process a sentence 
\begin{compactitem}
\item {\tt Manager.cpp} implements the decoding algorithm 

\item {\tt TranslationOptionCollection.cpp} contains translation options 

\item {\tt Hypothesis.cpp} represents partial translation 

\item {\tt HypothesisStack.cpp} contain viable hypotheses, implements pruning 

\end{compactitem}

\item Output results: {\tt moses-cmd/src/Main.cpp} 
\begin{compactitem}
\item {\tt moses-cmd/src/IOStream::OutputBestHypo} print best translation 

\item n-best lists generated in {\tt Manager.cpp}, output in {\tt IOStream.cpp}

\end{compactitem}

\end{compactitem}

\subsection{
Detailed Guides}

\begin{compactitem}
\item Style Guide (Section \ref{code-style.tex})

\item Background on factor, word, and phrase data structure (Section \ref{code-guide-factors.tex})

\item Chart decoder (Section \ref{code-guide-syntax.tex})

\item Multithreading (Section \ref{code-guide-threading.tex})

\item Adding feature functions (Section \ref{code-guide-ff.tex})

\end{compactitem}










\begin{center}\textit{\small Subsection last  modified on October 06, 2017, at 11:37 AM}\end{center}
