Building Moses Training Pipelines With Arrows
Project leader: Ian Johnson
Desirable skills for participants: Python
It is not about the choice of language (Perl/Python/Haskell?) to build training pipelines, but the choice of programming styles. Towards a plugable code structure, we propose to use Arrows to construct the whole training pipeline of Moses. Arrows are a programming construct that allow programmers to describe computations in a pure and declarative way. Arrows can be combined to produce new arrows which, themselves, can be combined with other arrows. Programs, including Moses training pipelines, are not simple functions from input to outputs. Since arrows allow for arbitrarily complex computations to be combined in many ways it possible to build "pluggable" and re-usable pipelines components.
Arrows also allow programmers to use a monadic programming style since arrows are more general than monads. Monads allow side-effects to be handled in a purely functional way and can make code simpler to follow since there is, usually, only one code path.
Haskell, a pure functional programming language, has implementations for arrows. However, the language is not widely known and can be intimidating to some. A sub-set of the arrows classes have been implemented in Python. Side effects are allowed in any part of a Python program so it is up to the programmer to choice a purely functional, monadic style, or side-effecting style: the Python implementation, therefore, supports both styles.
The aim of this MT marathon project is to assess the Python arrow implementation in these fours areas:
- correctness, - strength, - ease of use, and - usefulness
The method by which the arrow implementation shall be assessed is to attempt to replace the training pipeline implementation, in Moses today, with pluggable and re-usable arrow pipeline components.
Suggested Reading
Generalising Monads to Arrows, by John Hughes, Science of Computer Programming, 1998 Vol 37, pp 67-111
Arrows, By Christoph Galliker, June 2010
Kleisli arrows of outrageous fortune, Conor McBride, 2011