This is a generalization of the technique used in setuptools so people can write plugins for commands.

Let's take an example: you have a command where you create a list of files to build a file list (a manifest).

You provide a default system to build this list but you know some people will probably provide other strategies to build that list.

So let's declare a user option, called "manifest-makers", where an ordered list of plugins name can be declared.

We also declare a new attribute called extensible_options, to declare the list of options that are used to extend the command.

class MyCmd(Command):

    user_options = [('manifest-makers', None,
                     'Plugins to build the manifest file')]

    extensible_options = ['manifest-makers']

    def initialize_options(self):
        # this is a regular user option
        self.manifest_makers = ['svn', 'template', 'hg']
        self.files = []

    def finalize_options(self):

    def run(self):
        # this will build the filelist by running the plugins

What happened ? In the initialize options, we declared default values for the manifest_makers attribute : three plugins called 'svn', 'template' and 'hg'.

The Command will load these plugins using setuptools entry point called: "distutils.MyCmd.manifest_makers". It will load them at the end of the option finalization.

Then, a new API called "run_extension" allows MyCmd to run these plugins.

Each plugin receives the command and the name of the option in argument and is free to work over the command and its distribution.

For example, the signature for the svn plugin is :

def svn(cmd, name):
    # work done here on the command

Read more about how to create plugins with entry points, and what they are, here : http://lucumr.pocoo.org/2006/7/30/setuptools-plugins


Use cases

Some simple usecase, need solutions with the above design

Creating a command to build and install documentation

(by DavidCournapeau)

A python distribution package foo 1.0 is set-up as follows:


The documentation is in rest format and can be built by sphinx (e.g. (cd doc && make html)). The author wants to build the documentation automatically, and include it in a sdist-generated tarball. Two commands are needed: build_doc and install_doc.

Installing a C library meant to be used by other extensions

(by DavidCournapeau)

Example: in numpy, some core, portable mathematical routines are built in a pure C library (built through build_clib command). We want to install this library and makes it available to other python packages which are based on numpy. Problems:

Configuring external dependencies locations

(by DavidCournapeau)

Many python packages rely on some external libraries, often written in C/C++. How to detect them if they are installed in a non standard location ? In autoconf, there is a simple mechanism:

./configure --with-foo=/some/path 
(or ./configure --with-foo-include=/some/path/include --with-foo-lib=/some/path/lib)

and the foo header will be looked for in /some/path/include + /some/path/lib for the library. How to add those options to the config command ? How to pass the related information to other commands ?

Building a ctypes extension

(by DavidCournapeau)

A ctypes extension (i.e. a library which can be opened through dlopen/LoadLibrary/etc...) cannot be built with ctypes in a portable manner ATM. The problem is that some link options are different compared to a python extension (on windows in particular, where the symbols to export through /EXPORT are not the same). The compiler classes are too difficult to use/extend in their current state (they don't have the same interface, they sometimes fail for mysterious reasons, in particular the MSVCCompiler class which has not the same interface as the UnixCCompiler class).

How to write a build_ctypes command ?

Controlling compiler flags

(by DavidCournapeau)

Sometimes, it is needed to control compiler flags. For example, gcc offer a very useful variety of warning flags which are not always enabled, or we may want to add option for profiling/debugging/code coverage/etc... There is also the problem that different source files may need different compilation options.

Conditionally controlling compiler flags

(by FlorisBruynooghe)

As above, but only conditionally enable them with a command line option or in a configuration file. This way only a developer can easily enable flags like -Werror while a user won't be doing this. (Sorry it wasn't clear to me if the above included this or not)

Distutils/PluginSystem (last edited 2009-04-22 09:16:36 by tarek)

Unable to edit the page? See the FrontPage for instructions.