



\section{
\label{code-guide-sff.tex}
Adding Sparse Feature Functions}


Moses allows for sparse feature functions, i.e., feature functions that have a large, maybe unbounded, set of features, of which only a small subset applies to a given hypothesis. 



{\bf ALL} feature functions can contain sparse features. They don't have to specify if or how many sparse features they will have. Contrast this with dense features, where feature function must specify how many scores they have at construction time.



To give an example: In addition to a regular n-gram language model, we could introduce a discriminative bigram language model that discounts or promotes hypotheses that contain specific bigrams. Each bigram in this feature function is its own feature with its own feature weight. 



These features cannot be tuned with MERT, but Moses has several other suitable tuning (Section \ref{tuning.tex}) methods.



The incorporation of sparse features into the training pipeline\footnote{\sf http://www2.statmt.org/moses/?n=Moses.SparseFeatureTraining} is ongoing.



\subsection{
Implementation}


For basics, please refer to the respective section on Feature Functions (Section \ref{code-guide-ff.tex}).



\subsubsection*{
Header}

\begin{SaveVerbatim}{myverb}
class PhraseLengthFeature : public StatelessFeatureFunction {
public:
PhraseLengthFeature(const string &line):
StatelessFeatureFunction(0, line)
{}
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

This creates a feature function {\tt PhraseLengthFeature} with no dense features, but it can have sparse features.



\subsubsection*{
Setting feature values}


As with all feature functions, sparse feature functions should implement the appropriate {\tt Evaluate()} methods described in FeatureFunctions (Section \ref{code-guide-ff.tex}).



In the {\tt Evaluate()} methods, a particular sparse score can can set using by calling the function

\begin{SaveVerbatim}{myverb}
accumulator->PlusEquals(this, <name>, <value>);
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

where {\tt $<$name$>$} is a key (a string) of the sparse feature. 



Contrast this with setting a dense score:

\begin{SaveVerbatim}{myverb}
accumulator->PlusEquals(this, <vector-of-values>);
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

\subsection{
Weights}


There is no need to define a switch {\tt [weight]} for the feature function. Each feature of the feature function has its own named weight, which is a concatenation of the short name of the feature function, and underscore ({\tt \_}) and its individual name for which the feature function sets.



These weights are placed into a weight file which is specified with the switch {\tt -{}-weight-file}. For example, the target bigram feature weights (feature function short name {\tt dlmb} for discriminative language model, bigrams) may have weights defined in lines such this:






\begin{SaveVerbatim}{myverb}
dlmb_of:the 0.1
dlmb_in:the -0.1
dlmb_the:way 0.2
\end{SaveVerbatim}
\begin{shaded}
\small
\UseVerbatim{myverb}
\end{shaded}

Features that do not have weights that are defined in this file are set to 0.












\begin{center}\textit{\small Subsection last  modified on December 06, 2013, at 03:15 PM}\end{center}
