Enemies of Carlotta/ development

EoC development branch

EoC development happens in two main branches: the "stable" one (1.2.x at the moment) and the "development" branch. The stable branch tries to avoid problems, at the expense of new functionality. The development branch embraces new functionality, at the possible expense of stability.

Currently, EoC 2 is being developed as the development branch. No useful releases have been made yet. This page acts as its design document. For discussion, please see the mailing list.

See branches for the list of Bazaar branches related to EoC.

Requirements

There are probably more requirements. Feel free to suggest them.

Overview

The design is called "do-nothing". The core of the new system will do nothing, except load plugins, read configuration files, parse the command line, and provide the main line of processing of e-mails. All other aspects of the program will happen in plugins: reading e-mails, sending e-mails, managing a subscriber database, handling bounces, etc.

Plugins

A plugin is a class that inherits the eoc.Plugin class. Plugins are loaded from .py files located in dedicate plugin directories:

The default list can be overridden by setting the EOC_PLUGIN_DIRS environment variable, or via the configuration files or command line options.

The EoC core will load plugin files and instantiate all classes they contain which inherit the eoc.Plugin class.

A plugin file MUST NOT have any side effects from being loaded. This is necessary for unit testing to work: it is possible the plugin files will be loaded many times.

Likewise, the Plugin sub-class must not have any side effects from being instantiated.

Side effect in this context means any change to the state of the system which changes how it works. For example, creating a file, opening a network connection, and modifying configuration are all side effects. Writing to the log file is not a side effect.

Side effects should happen when the appropriate hooks are called. For example, there will be a hook like "setup-plugins" which can cause any desired external initialization.

There are only one kinds of plugins. There will be all sorts of hooks, and plugins can do different things depending on which hook they attach to, so basically any plugin can do anything.

There will probably be plugins that, for example, deal only with sending mail (SMTP, /usr/sbin/sendmail, QMQP, and maybe others), or only with storing the subscriber list in an SQL database, or only deal with archiving, or only deal with spam prevention. So in that way there will be different kinds of plugins, but it won't be enforced by the design or the implementation. No plugin needs to attach to every hook.

Hooks

The main line of processing provides some "hooks" which plugins can hang onto. For example, there will be a read-email hook, which will be called when the main line of processing is ready to receive an e-mail. Different plugins will do the reading in different ways. One plugin might read it from the standard input, another might read it from a Maildir, and a third one might read it from an SQL database.

Hooks have names, and plugins may register themselves as callbacks for them. Then, depending on circumstances, one or more of the callbacks may be called. For the read-email hook, the first hook to actually return a new message will be the last one called. Other hooks may call all callbacks (in random order) and then use the result. For example, a hook for modifying a message that is being sent out might be such a hook.

Registrations and callbacks will happen in an undefined order. Plugins MUST NOT assume that either order is deterministic.

The list of hooks will be determined during the implementation phase. The goal is to be generous with hooks: anything that can reasonably be affected by a plugin should have a hook.

Configuration files and command line processing

EoC core is in charge of loading configuration files and processing command line arguments. Plugins MUST NOT store configuration elsewhere, for the sake of the sanity of listmasters.

There is a list of configuration files which will be read by default, if they exist (this list is probably subject to change soon enough):

The first one is meant to be installed by the package manager and provide defaults, which can then be overridden by the system administrator or listmaster. This setup avoids unnecessary configuration file prompting when upgrading by package managers such as dpkg.

The last file on the list contains list-specific configuration. It is loaded after the name of the list in question is determined.

The configuration defined by the configuration files consists of key, value pairs. Configuration files follow the ini-file syntax implemented by Python's ConfigParser class, but only the eoc section is used.

EoC core provides a set of configuration keys, and plugins may add more.

Command line options override configuration file settings. Each configuration key corresponds to a long option.

Example: ~/.enemies-of-carlotta/eoc.conf:

[eoc]
log-file = ~/.enemies-of-carlotta/log/eoc.log
log-level = info
incoming = ~/.enemies-of-carlotta/Maildir
send-mail = smtp

Example: ~/.enemies-of-carlotta/eoc@liw.iki.fi/config:

[eoc]
owners = liw@iki.fi
posting = subscriber-only

It would also be possible to set the owner in the eoc.conf file, if it is the same for all or most lists, and then override it for the exceptions.

There is no per-plugin configuration file. There is no configuration saying which plugins should be loaded for which list: they are all loaded always, and will use the configuration data to determine whether they're wanted. For example, there might be a configuration variable called mail-source which is set to stdin, Maildir, or SQL. The plugins that hook into the read-email hook look at the value of mail-source and determine if they're wanted or not.

If a configuration variable does not explicitly trigger a plugin, the plugin should do nothing.