Approach 1: Maintain a Python 2 base and use 2to3 to generate Python 3 code

Make sure the code runs in Python 2.6 and use 2to3

This approach does not require that support for versions before 2.6 needs to be dropped. Even the need to test with 2.6 is not strict - one could just as well use this approach on code that has never been tested with 2.6, and is only known to work on 2.5.

2to3 is a Python program that reads Python 2.x source code and applies a series of fixers to transform it into valid Python 3.x code. The standard library contains a rich set of fixers that will handle almost all code. 2to3 supporting library lib2to3 is, however, a flexible and generic library, so it is possible to write your own fixers for 2to3. lib2to3 could also be adapted to custom applications in which Python code needs to be edited automatically. For more information about 2to3, see: http://doc.python.org/library/2to3.html

Usage of 2to3 can be integrated into the installation process: 2to3 can be run as a build step in distutils. With distutils, the following fragments are needed in a setup.py:

try:
    from distutils.command.build_py import build_py_2to3 as build_py
except ImportError:
    # 2.x
    from distutils.command.build_py import build_py
...
setup(...
  cmdclass = {'build_py':build_py}
)

This will leave intact all source files, and convert them to Python 3 in the build area, if setup.py is run on Python 3. 'setup.py install' will then copy the 3.x version of the code into the target directory. If run on 2.x, nothing will change at all (as 2.x doesn't provide the build_py_2to3 class).

For distribute, this approach can be expressed as:

setup(...
  use_2to3=True
)

Manual changes (not done by 2to3):

• os.path.walk => os.walk: see issue4601.
• rfc822 => email

Additional resources:

• http://diveintopython3.org/porting-code-to-python-3-with-2to3.html
• http://packages.python.org/distribute/python3.html
• http://lucumr.pocoo.org/2010/2/11/porting-to-python-3-a-guide
• http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/
• http://techspot.zzzeek.org/2011/01/24/zzzeek-s-guide-to-python-3-porting
• http://dabeaz.blogspot.com/2011/01/porting-py65-and-my-superboard-to.html
• http://python3porting.com/strategies.html
• http://lwn.net/Articles/426906/

sys.stdout.flush()
sys.stdout.buffer.write(some_binary_data)

Approach 2: Make code run unmodified in both Python 2 and Python 3

Supporting code that runs in both Python 2.6 and Python 3 is not much more difficult than porting to Python 3 (see above for more details). However, supporting older versions--like 2.3, 2.4, and 2.5--is much more difficult. This requires avoiding any syntax that only works in one version or the other, so it often involves resorting to hacks. These hacks make the code more awkward than it would be in Python 3 or in Python 2, but many people find that this is better than supporting only one version or the other. When support for Python 2 is eventually dropped, these hacks can be removed.

Python 2.6 introduces forward-compatibility for many new Python 3 features. The Python 3 builtins are accessible by doing an import from future_builtins. To use the new print function, use from __future__ import print_function, and to make string literals be interpreted as unicode, use from __future__ import unicode_literals. These forward-compatibility features, and many others, are described in greater detail in What's New in Python 2.6:

Additional resources:

• http://pydev.blogspot.com/2008/11/making-code-work-in-python-2-and-3.html
• http://mail.mems-exchange.org/durusmail/qp/441/
• http://mikewatkins.ca/2008/11/29/python-2-and-3-metaclasses/

Six is a library which has helpers to simplify writing codebases that support both Python 2 and 3. See http://pypi.python.org/pypi/six.

try:
  bytes # Forward compatibility with Py3k
except NameError:
  bytes = str

try:
  str = unicode
except NameError:
  pass # Forward compatibility with Py3k

print 'hello world'

import sys
sys.stdout.write('hello world\n')

print('hello world')

import sys
try:
    open('/path/to/some/file')
except IOError:
    _, e, _ = sys.exc_info()

from xyz import abc

from mypackage.xyz import abc

Approach 3: Maintain Separate Releases for Python 2 and Python 3

Convert a Python 2 tree to Python 3 with the 2to3 tool: http://doc.python.org/library/2to3.html

Martin's notes from psycopg2

• the buffer object is gone; I use memoryview in 3.x.
• various tests where in the code of the form if version_major == and version_minor > 4 (say, or 5) This will break for 3.x; you have to write if (version_major == 2 and version_minor > 4) or version_major > 2
• Python code 2: setup.py needs to run in both versions. I had to replace popen2 with subprocess if available. Also, map() now returns an iterator, which I explicitly convert into list, and so on.
• Python code 3: the test suite doesn't get installed, and hence not auto-converted with 2to3 support. I explicitly added a 2to3 conversion into the test runner, which copies the py3 version of the test into a separate directory.

Differences in specific libraries

These are porting notes on differences in third party libraries when run on Python 2 and Python 3.

• PyQt4

Links

• a bunch of very useful reference code: http://code.google.com/p/python-incompatibility/
• PortingDjangoTo3k
• Porting Durus / QP / Qpy / Dulcinea, using approach targeting Python 2 and Python 3 off same code base - packages released but not in pypi yet.
• Early2to3Migrations
• Stephan Deibel's blog post on code usable from Python 2.0 through 3.0
• Python code that demonstrates differences between 2.5 and 3.0, and ways of making the same code run on 2.6 and 3.x
• Georg Brandl's blog post collecting some information: http://pythonic.pocoo.org/2008/12/14/python-3-porting-resources
• A blog post about porting experience: psycopg2 porting to Python 3: a report
• Reference Card: Moving from Python 2 to Python 3: http://ptgmedia.pearsoncmg.com/imprint_downloads/informit/promotions/python/python2python3.pdf
• Lennart Regebro's overview of Porting strategies