API

The proposed extension would consist of the following objects:

engine.ImportEngine

__import__(self, name, globals={}, locals={}, fromlist=[], level=0) Reimplementation of the builtin __import__() function. The import of a module will proceed using the state stored in the ImportEngine instance rather than the global import state. For full documentation of __import__ funtionality, see [2] . __import__() from ImportEngine and its subclasses can be used to customise the behaviour of the import statement by replacing __builtin__.__import__ with ImportEngine.__import__.

import_module(name, package=None)

A reimplementation of importlib.import_module() which uses the import state stored in the ImportEngine instance. See [3] for a full reference.

from_engine(self, other)

Create a new import object from another ImportEngine instance. The new object is initialised with a copy of the state in other. When called on engine.sysengine as other, from_engine() can be used to create an ImportEngine object with a copy of the global import state.

GlobalImportEngine(ImportEngine)

Convenience class to provide engine-like access to the global state. Provides __import__(), import_module() and from_engine() methods like ImportEngine but writes through to the global state in sys.

Global variables

engine.sysengine

Instance of GlobalImportEngine provided for convenience (e. g. for use by module finders and loaders).

Necessary changes to finder/loader interfaces:

find_module (cls, fullname, path=None, engine=None)

load module (cls, fullname, path=None, engine=None)

The only difference between 'new style' and PEP 302 compatible finders/loaders is the presence of an additional engine parameter. This is intended to specify an ImportEngine instance or subclass there of. This parameter is optional so that the 'new style' finders and loaders can be made backwards compatible by falling back on engine.sysengine with the following simple pattern:

find_module(cls, fullname, path=None, engine=None)
  if not engine:
    engine = engine.sysengine

  ...

An implementation based on Brett Cannon's importlib has been developed by Greg Slodkowicz as part of the 2011 Google Summer of Code. The code repository is located at https://bitbucket.org/jergosh/gsoc_import_engine/.

Open Issues

The existing importlib implementation depends on several functions from imp, Python's builtin implementation of __import__ located in Python/import.c. These functions are unaware of ImportEngine and place the newly imported module in sys.modules. Naturally, this is a problem from the ImportEngine point of view. The offending methods are:

• imp.init_builtin()
• imp.load_dynamic()

However, since there can be only a single instance of each builtin/dynamic module per process, they are essentially process-global regardless of the way they are imported. Currently, the simplest solution for supporting them in ImportEngine seems to have new style loaders call the existing imp methods and then copy appropriate references from sys.modules into the state inside the import engine.

Similarly, imp.NullImporter implements a load_module method which is incompatible with 'new style' loaders. Since the NullImporter class does next to nothing (i. e. always returns None), it has been reimplemented in Python. The only way this could cause problems would be explicitly checking if a module's importer is an imp.NullImporter (which occurs only in some unittests).