Extending Sumatra with plug-ins¶
To support the wide diversity of scientific workflows, Sumatra aims to make it easy (or at least possible) to customize the way it works. The following components can be customized:
- launching computations (local execution, distributed using MPI, batch jobs using SLURM, ...)
- parameter file formats (JSON, YAML, ...)
- executable programs/programming languages (Python, MATLAB, R, ...)
- data stores (local disk, network storage, WebDAV, Dropbox, ...)
- record stores (SQLite, PostgreSQL, web-based, ...)
- output formats (plain text, HTML, LaTeX, JSON, ...)
- version control systems (Git, Mercurial, Subversion, ...)
Here is an example of wrapping the PHP command-line interface. (In itself, this gives little or no benefit, as Sumatra can automatically obtain version information from most command-line tools, but (a) it is a prerequisite if we wish to add dependency tracking, (b) it makes a nicely simple example of writing a plug-in!)
from sumatra.core import component from sumatra.programs import Executable @component class PHPcli(Executable): name = "PHP" executable_names = ('php',) file_extensions = ('.php',) default_executable_name = 'php' requires_script = True
To use this plug-in in a Sumatra project, it should be saved in a file (e.g.
that is on the Python module search path (for example, installed in
added to the
PYTHONPATH environment variable), and then added to the project using:
$ smt configure --add-plugin=php
You can also manually edit
.smt/project and add the module path to the “plugins” list.
To list the installed plug-ins, use
To remove a plug-in from a project:
$ smt configure --remove-plugin=php
As shown in the PHP example above, writing a custom component involves writing a new class which inherits from a Sumatra base class, and then registering the class with the component registry.
The available base classes are:
For full details of which methods need to be implemented in your sub-class, see the API reference.
At present, there is no mechanism for plug-ins to change the list of options
for command-line tools such as
smt. This is planned for the future.