|
Size: 2084
Comment: workin on it
|
Size: 5378
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| == example == | = Example = This is a generalization of the technique used in setuptools so people can write plugins for commands. Let's take an example: you have a command where you create a list of files to build a file list (a manifest). You provide a default system to build this list but you know some people will probably provide other strategies to build that list. |
| Line 4: | Line 13: |
| So let's declare a user option, called "manifest-makers", where an ordered list of plugins name can be declared. We also declare a new attribute called `extensible_options`, to declare the list of options that are used to extend the command. |
|
| Line 5: | Line 20: |
| class MyCmd(sdist): | class MyCmd(Command): |
| Line 7: | Line 22: |
| # this is a regular user option manifest_makers = ['svn', 'template', 'hg'] |
user_options = [('manifest-makers', None, 'Plugins to build the manifest file')] |
| Line 10: | Line 25: |
| # this manages all extensions extensions = Extensible('manifest_makers') |
extensible_options = ['manifest-makers'] |
| Line 13: | Line 27: |
| def run(self): # this will build the filelist by running the plugins self.extensions.run('manifest_makers') |
def initialize_options(self): # this is a regular user option self.manifest_makers = ['svn', 'template', 'hg'] self.files = [] |
| Line 17: | Line 32: |
| def finalize_options(self): pass |
|
| Line 18: | Line 35: |
| dist = Distribution() cmd = MyCmd(dist) cmd.ensure_finalized() cmd.run() |
def run(self): # this will build the filelist by running the plugins self.run_extension('manifest-makers') |
| Line 25: | Line 41: |
| Notice that self.extensions.load(self) is implicitely called by Command. | What happened ? In the initialize options, we declared default values for the manifest_makers attribute : three plugins called 'svn', 'template' and 'hg'. |
| Line 27: | Line 44: |
| == code == | The Command will load these plugins using setuptools entry point called: "distutils.!MyCmd.manifest_makers". It will load them at the end of the option finalization. Then, a new API called "run_extension" allows !MyCmd to run these plugins. Each plugin receives the command and the name of the option in argument and is free to work over the command and its distribution. For example, the signature for the svn plugin is : {{{ def svn(cmd, name): # work done here on the command }}} Read more about how to create plugins with entry points, and what they are, here : http://lucumr.pocoo.org/2006/7/30/setuptools-plugins = Implementation = * http://svn.plone.org/svn/collective/collective.releaser/branches/refactor/collective/releaser/commands/extendable.py * http://svn.plone.org/svn/collective/collective.releaser/branches/refactor/collective/releaser/tests/test_extendable.py = Use cases = Some simple usecase, need solutions with the above design == Creating a command to build and install documentation == (by DavidCournapeau) A python distribution package foo 1.0 is set-up as follows: |
| Line 30: | Line 76: |
| """Extendable command""" import os import pkg_resources |
foo-1.0/setup.py foo/__init__.py foo/.. doc/ }}} |
| Line 34: | Line 82: |
| class Extensible(object): | The documentation is in rest format and can be built by sphinx (e.g. (cd doc && make html)). The author wants to build the documentation automatically, and include it in a sdist-generated tarball. Two commands are needed: build_doc and install_doc. |
| Line 36: | Line 84: |
| def __init__(self, *options): self.options = options self._ext = {} |
== Installing a C library meant to be used by other extensions == |
| Line 40: | Line 86: |
| def load(self, cmd): for opt in self.options: ep = ExtensionPoint(cmd, opt) ep.load() self._ext[opt] = ep |
(by DavidCournapeau) |
| Line 46: | Line 88: |
| def run(self, name): self._ext[name].run() |
Example: in numpy, some core, portable mathematical routines are built in a pure C library (built through build_clib command). We want to install this library and makes it available to other python packages which are based on numpy. Problems: |
| Line 49: | Line 90: |
| class ExtensionPoint(object): | * how to install it ? The usual solution is to handle this in the install command. Only build_clib knows where the library is built in the build directory (which is platform dependent and hardcoded in the build_clib command), so the install command has no way to know the location without hacking more into communication between both commands. More fundamentally, it seems very complicated to extend commands to just install one file. * how to install the corresponding library API declaration (the .h file) in a known location (could be done through install_data, I guess). |
| Line 51: | Line 93: |
| def __init__(self, cmd, name): self.name = name self.cmd = cmd self.entries = [] |
== Configuring external dependencies locations == |
| Line 56: | Line 95: |
| def load(self, name=None): if name is None: name = self.cmd.get_command_name() entry_point = 'distutils.%s:%s' % (name, self.name) else: entry_point = name self._entries = [entry for entry in [self._load(ep) for ep in pkg_resources.iter_entry_points(entry_point)] if entry is not None] |
(by DavidCournapeau) |
| Line 66: | Line 97: |
| def _load(self, entry_point): values = getattr(self.cmd, self.name) if not isinstance((list, tuple), values): values = [values] entry_point = entry_point.load() if entry_point.name not in values: return None return entry_point |
Many python packages rely on some external libraries, often written in C/C++. How to detect them if they are installed in a non standard location ? In autoconf, there is a simple mechanism: |
| Line 75: | Line 99: |
| def run(self): for ep in self._entries: entry_point(self.cmd, self.name) |
{{{ ./configure --with-foo=/some/path (or ./configure --with-foo-include=/some/path/include --with-foo-lib=/some/path/lib) }}} |
| Line 79: | Line 104: |
| def apply(self, name=None): self.load(name) self.run() }}} |
and the foo header will be looked for in /some/path/include + /some/path/lib for the library. How to add those options to the config command ? How to pass the related information to other commands ? == Building a ctypes extension == (by DavidCournapeau) A ctypes extension (i.e. a library which can be opened through dlopen/LoadLibrary/etc...) cannot be built with ctypes in a portable manner ATM. The problem is that some link options are different compared to a python extension (on windows in particular, where the symbols to export through /EXPORT are not the same). The compiler classes are too difficult to use/extend in their current state (they don't have the same interface, they sometimes fail for mysterious reasons, in particular the MSVCCompiler class which has not the same interface as the UnixCCompiler class). How to write a build_ctypes command ? == Controlling compiler flags == (by DavidCournapeau) Sometimes, it is needed to control compiler flags. For example, gcc offer a very useful variety of warning flags which are not always enabled, or we may want to add option for profiling/debugging/code coverage/etc... There is also the problem that different source files may need different compilation options. |
Example
This is a generalization of the technique used in setuptools so people can write plugins for commands.
Let's take an example: you have a command where you create a list of files to build a file list (a manifest).
You provide a default system to build this list but you know some people will probably provide other strategies to build that list.
So let's declare a user option, called "manifest-makers", where an ordered list of plugins name can be declared.
We also declare a new attribute called extensible_options, to declare the list of options that are used to extend the command.
class MyCmd(Command):
user_options = [('manifest-makers', None,
'Plugins to build the manifest file')]
extensible_options = ['manifest-makers']
def initialize_options(self):
# this is a regular user option
self.manifest_makers = ['svn', 'template', 'hg']
self.files = []
def finalize_options(self):
pass
def run(self):
# this will build the filelist by running the plugins
self.run_extension('manifest-makers')What happened ? In the initialize options, we declared default values for the manifest_makers attribute : three plugins called 'svn', 'template' and 'hg'.
The Command will load these plugins using setuptools entry point called: "distutils.MyCmd.manifest_makers". It will load them at the end of the option finalization.
Then, a new API called "run_extension" allows MyCmd to run these plugins.
Each plugin receives the command and the name of the option in argument and is free to work over the command and its distribution.
For example, the signature for the svn plugin is :
def svn(cmd, name):
# work done here on the commandRead more about how to create plugins with entry points, and what they are, here : http://lucumr.pocoo.org/2006/7/30/setuptools-plugins
Implementation
Use cases
Some simple usecase, need solutions with the above design
Creating a command to build and install documentation
(by DavidCournapeau)
A python distribution package foo 1.0 is set-up as follows:
foo-1.0/setup.py
foo/__init__.py
foo/..
doc/The documentation is in rest format and can be built by sphinx (e.g. (cd doc && make html)). The author wants to build the documentation automatically, and include it in a sdist-generated tarball. Two commands are needed: build_doc and install_doc.
Installing a C library meant to be used by other extensions
(by DavidCournapeau)
Example: in numpy, some core, portable mathematical routines are built in a pure C library (built through build_clib command). We want to install this library and makes it available to other python packages which are based on numpy. Problems:
- how to install it ? The usual solution is to handle this in the install command. Only build_clib knows where the library is built in the build directory (which is platform dependent and hardcoded in the build_clib command), so the install command has no way to know the location without hacking more into communication between both commands. More fundamentally, it seems very complicated to extend commands to just install one file.
- how to install the corresponding library API declaration (the .h file) in a known location (could be done through install_data, I guess).
Configuring external dependencies locations
(by DavidCournapeau)
Many python packages rely on some external libraries, often written in C/C++. How to detect them if they are installed in a non standard location ? In autoconf, there is a simple mechanism:
./configure --with-foo=/some/path (or ./configure --with-foo-include=/some/path/include --with-foo-lib=/some/path/lib)
and the foo header will be looked for in /some/path/include + /some/path/lib for the library. How to add those options to the config command ? How to pass the related information to other commands ?
Building a ctypes extension
(by DavidCournapeau)
A ctypes extension (i.e. a library which can be opened through dlopen/LoadLibrary/etc...) cannot be built with ctypes in a portable manner ATM. The problem is that some link options are different compared to a python extension (on windows in particular, where the symbols to export through /EXPORT are not the same). The compiler classes are too difficult to use/extend in their current state (they don't have the same interface, they sometimes fail for mysterious reasons, in particular the MSVCCompiler class which has not the same interface as the UnixCCompiler class).
How to write a build_ctypes command ?
Controlling compiler flags
(by DavidCournapeau)
Sometimes, it is needed to control compiler flags. For example, gcc offer a very useful variety of warning flags which are not always enabled, or we may want to add option for profiling/debugging/code coverage/etc... There is also the problem that different source files may need different compilation options.