Extending Sumatra with plug-ins

To support the wide diversity of scientific workflows, Sumatra aims to make it easy (or at least possible) to customize the way it works. The following components can be customized:

  • launching computations (local execution, distributed using MPI, batch jobs using SLURM, …)

  • parameter file formats (JSON, YAML, …)

  • executable programs/programming languages (Python, MATLAB, R, …)

  • data stores (local disk, network storage, WebDAV, Dropbox, …)

  • record stores (SQLite, PostgreSQL, web-based, …)

  • output formats (plain text, HTML, LaTeX, JSON, …)

  • version control systems (Git, Mercurial, Subversion, …)

Here is an example of wrapping the PHP command-line interface. (In itself, this gives little or no benefit, as Sumatra can automatically obtain version information from most command-line tools, but (a) it is a prerequisite if we wish to add dependency tracking, (b) it makes a nicely simple example of writing a plug-in!)

from sumatra.core import component
from sumatra.programs import Executable

@component
class PHPcli(Executable):
    name = "PHP"
    executable_names = ('php',)
    file_extensions = ('.php',)
    default_executable_name = 'php'
    requires_script = True

To use this plug-in in a Sumatra project, it should be saved in a file (e.g. php.py) that is on the Python module search path (for example, installed in site-packages or added to the PYTHONPATH environment variable), and then added to the project using:

$ smt configure --add-plugin=php

You can also manually edit .smt/project and add the module path to the “plugins” list.

To list the installed plug-ins, use smt info.

To remove a plug-in from a project:

$ smt configure --remove-plugin=php

Component interfaces

As shown in the PHP example above, writing a custom component involves writing a new class which inherits from a Sumatra base class, and then registering the class with the component registry.

The available base classes are:

For full details of which methods need to be implemented in your sub-class, see the API reference.

Note

At present, there is no mechanism for plug-ins to change the list of options for command-line tools such as smt. This is planned for the future.