Cruise Control for Moses

Participants: Ondrej Bojar, Barry Haddow, Ales Tamchyna, Rimas Blazaitis

The goal of the project is to implement a simple framework for regular testing (aka cruise control). The very basic functionality (checkout, configure according to the user's setting, compilation, logging) is already implemented in moses/cruise-control. The aims now are:

  • add regression tests to the cruise control script (3-30 lines of bash :-)
  • deploy testing at a few sites (of project members of anyone interested in having his particular Moses setup tested every day) (1 cron line :-)

These are valuable extensions:

  • (also see cruise-control/README)
  • make sure the configurability of MCC is sufficient for all users interested in their various Moses setups
  • add web presentation of the logs
    • e.g. a PHP script sitting next to the logs directory and making a table of the brief log and links to the detailed logs
    • or a Perl script to prepare this summary off-line at the end of each cruise control run (maybe preferred)
  • visualization (in the web presentation)
    • the cruise control is based on git view of Moses code, so the visualization could have the form of the beautiful DAG as you know from gitk, github or (console-based) tig
      • the nodes(=commits) should go from red to green depending on how far did the cruise control successfully get
      • pointing at the node should say a bit more, clicking should show the log etc.
  • joint reporting from various sites:
    • e.g. commit also the brief logs to the repository
    • add a presentation layer that merges all brief logs to a single table listing how many test sites in how many configs succeeded with each particular commit (including the beautiful visualization)
    • reporting based on e.g. configs or environment as logged in the detailed logs: if I want IrstLM on 32bit machines with a particular distro, how likely based on other cruise control sites am I to succeed, or rather: which commit should I use to have no troubles.
  • other summaries, e.g. top committers that break/fix it
  • Extend Moses test framework, to make it easier to test the components of the training pipeline, and to test the pipeline as a whole.

Project proposed by Ondrej Bojar (a remote leader who may unfortunately even fail to get online throughout the week; roaming is unreliable).

Project leaders on site: Barry Haddow or anybody else

Page last modified on September 06, 2011, at 03:45 PM