Cruise Control for Moses
Participants: Ondrej Bojar, Barry Haddow, Ales Tamchyna, Rimas Blazaitis
The goal of the project is to implement a simple framework for regular
testing (aka cruise control). The very basic functionality (checkout,
configure according to the user's setting, compilation, logging) is
already implemented in moses/cruise-control. The aims now are:
- add regression tests to the cruise control script (3-30 lines of bash :-)
- deploy testing at a few sites (of project members of anyone interested in having his particular Moses setup tested every day) (1 cron line :-)
These are valuable extensions:
- (also see cruise-control/README)
- make sure the configurability of MCC is sufficient for all users interested in their various Moses setups
- add web presentation of the logs
- e.g. a PHP script sitting next to the logs directory and making a table of the brief log and links to the detailed logs
- or a Perl script to prepare this summary off-line at the end of each cruise control run (maybe preferred)
- visualization (in the web presentation)
- the cruise control is based on git view of Moses code, so the visualization could have the form of the beautiful DAG as you know from gitk, github or (console-based) tig
- the nodes(=commits) should go from red to green depending on how far did the cruise control successfully get
- pointing at the node should say a bit more, clicking should show the log etc.
- joint reporting from various sites:
- e.g. commit also the brief logs to the repository
- add a presentation layer that merges all brief logs to a single table listing how many test sites in how many configs succeeded with each particular commit (including the beautiful visualization)
- reporting based on e.g. configs or environment as logged in the detailed logs: if I want IrstLM on 32bit machines with a particular distro, how likely based on other cruise control sites am I to succeed, or rather: which commit should I use to have no troubles.
- other summaries, e.g. top committers that break/fix it
- Extend Moses test framework, to make it easier to test the components of the training pipeline, and to test the pipeline as a whole.
Project proposed by Ondrej Bojar (a remote leader who may unfortunately
even fail to get online throughout the week; roaming is unreliable).
Project leaders on site: Barry Haddow or anybody else