lm::builder::CorpusCount Class Reference

#include <corpus_count.hh>

Collaboration diagram for lm::builder::CorpusCount:

Collaboration graph

List of all members.

Public Member Functions

 CorpusCount (util::FilePiece &from, int vocab_write, uint64_t &token_count, WordIndex &type_count, std::vector< bool > &prune_words, const std::string &prune_vocab_filename, std::size_t entries_per_block, WarningAction disallowed_symbol)
void Run (const util::stream::ChainPosition &position)

Static Public Member Functions

static float DedupeMultiplier (std::size_t order)
static std::size_t VocabUsage (std::size_t vocab_estimate)

Detailed Description

Definition at line 23 of file corpus_count.hh.

Constructor & Destructor Documentation

lm::builder::CorpusCount::CorpusCount ( util::FilePiece from,
int  vocab_write,
uint64_t &  token_count,
WordIndex type_count,
std::vector< bool > &  prune_words,
const std::string &  prune_vocab_filename,
std::size_t  entries_per_block,
WarningAction  disallowed_symbol 

Definition at line 162 of file corpus_count.cc.

Member Function Documentation

float lm::builder::CorpusCount::DedupeMultiplier ( std::size_t  order  )  [static]

Definition at line 154 of file corpus_count.cc.

void lm::builder::CorpusCount::Run ( const util::stream::ChainPosition position  ) 

std::size_t lm::builder::CorpusCount::VocabUsage ( std::size_t  vocab_estimate  )  [static]

Definition at line 158 of file corpus_count.cc.

The documentation for this class was generated from the following files:

Generated on Thu Jul 6 00:32:43 2017 for Moses by  doxygen 1.5.9