11334
Comment: Numpy 1.5 supports Python 3
|
23823
Stronger statement/more information on Py2 sunset
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
#acl Known:read,write,revert All:read |
|
Line 5: | Line 3: |
You should use Python 3 going forward. As of January 2020 [[https://www.python.org/dev/peps/pep-0373/|Python 2 will be in EOL (End Of Life) status]] and receive no further official support.After that date, there will be no further updates nor bugfixes. Since this end-of-life date has been planned for nearly a decade (the first end-of-life date was slated to happen in 2014, and was pushed back to 2020), and nearly all popular libraries [[https://python3statement.org/|have already ported their code]], Python 2.x is well on its way to obsolescence. As such, we can only recommend learning and teaching Python 3 - which is simpler and will remain relevant well into the future. This page is largely out of date as of October 2019, but provides a historical reference as it provides value for maintaining legacy Python 2 systems. <<TableOfContents()>> |
|
Line 6: | Line 10: |
''Short version: Python 2.x is legacy, Python 3.x is the present and future of the language'' | |
Line 7: | Line 12: |
''Short version: Python 2.x is the status quo, Python 3.x is the shiny new thing.'' | Python 3.0 was released in 2008. The final 2.x version 2.7 release came out in mid-2010, with a statement of extended support for this end-of-life release. The 2.x branch will see no new major releases after that. 3.x is under active development and has already seen over five years of stable releases, including version 3.3 in 2012, 3.4 in 2014, 3.5 in 2015, and 3.6 in 2016. This means that all recent standard library improvements, for example, are only available by default in Python 3.x. |
Line 9: | Line 14: |
At the time of writing (July 4, 2010), the final 2.7 release is out, with a statement of extended support for this end-of-life release. The 2.x branch will see no new major releases after that. 3.x is under active and continued development, with 3.1 already available and 3.2 due for release around the turn of the year. | Guido van Rossum (the original creator of the Python language) decided to clean up Python 2.x properly, with less regard for backwards compatibility than is the case for new releases in the 2.x range. The most drastic improvement is the better Unicode support (with all text strings being Unicode by default) as well as saner bytes/Unicode separation. |
Line 11: | Line 16: |
3.x is the newest branch of Python and the intended future of the language. Guido van Rossum (the original creator of the Python language) decided to clean up Python 2.x properly, with less regard for backwards compatibility than is the case for new releases in the 2.x range. This allowed several aspects of the core language (such as print and exec being statements, integers using floor division) to be adjusted to be easier for newcomers to learn and to be more consistent with the rest of the language. It also allowed later language features (such as iterators) to be applied to older language features (such as the `range` builtin which returns a list in 2.x, but an iterator in 3.x). | Besides, several aspects of the core language (such as print and exec being statements, integers using floor division) have been adjusted to be easier for newcomers to learn and to be more consistent with the rest of the language, and old cruft has been removed (for example, all classes are now new-style, "range()" returns a memory efficient iterable, not a list as in 2.x). |
Line 13: | Line 18: |
The [[http://docs.python.org/release/3.1.2/whatsnew/3.0.html|What's New in Python 3.0]] document provides a good overview of the major language changes and likely sources of incompatibility with existing Python 2.x code. | The [[http://docs.python.org/py3k/whatsnew/3.0.html|What's New in Python 3.0]] document provides a good overview of the major language changes and likely sources of incompatibility with existing Python 2.x code. Nick Coghlan (one of the CPython core developers) has also created a [[http://python-notes.curiousefficiency.org/en/latest/python3/questions_and_answers.html|relatively extensive FAQ]] regarding the transition. |
Line 15: | Line 20: |
However, the broader Python ecosystem has amassed a significant amount of quality software over the years. The downside of breaking backwards compatibility in 3.x is that a lot of that software doesn't work on 3.x yet. | However, the broader Python ecosystem has amassed a significant amount of quality software over the years. The downside of breaking backwards compatibility in 3.x is that some of that software (especially in-house software in companies) still doesn't work on 3.x yet. |
Line 17: | Line 22: |
== So which version should I use? == |
== Which version should I use? == |
Line 21: | Line 25: |
If you can do exactly what you want with Python 3.x, great! There's a few downsides, such as comparatively limited library support and the fact that current Linux distributions and Macs are still shipping with 2.x by default, but as a language Python 3.x is definitely ready. As long as actually getting Python 3.x on your user's computers (which ought to be easy since a lot of people reading this may only be developing something for themselves or an environment they control) and you're writing things where lack of third party software isn't a major impediment (or where you know the packages you need already support Python 3), Python 3.x is an excellent choice. Also, quite a few distributions have Python 3.x available already for end-users, even if none of them are using it as the default interpreter at the moment. | If you can do exactly what you want with Python 3.x, great! There are a few minor downsides, such as very slightly worse library support<<FootNote(Amongst still-maintained packages: https://python3wos.appspot.com)>> and the fact that some current Linux distributions and Macs still use 2.x as default, but as a language Python 3.x is definitely mature and ready for use. As long as Python 3.x is installed on your user's computers (which ought to be easy, since many people reading this may only be developing something for themselves or an environment they control) and you are writing things where you know none of the Python 2.x modules are needed, it is an excellent choice. Also, most Linux distributions have Python 3.x already installed, and nearly all have it available for end-users. One somewhat painful exception is Red Hat Enterprise Linux through version 7: Python 3 does exist in the EPEL repository, but some users may not be allowed by company policy to install anything from such add-on locations - see note on "control" below. Some distributions are phasing out Python 2 as preinstalled default.<<FootNote(Arch Linux links python to python3, and Ubuntu and Fedora switch defaults: https://wiki.ubuntu.com/Python/3 https://fedoraproject.org/wiki/Changes/Python_3_as_Default)>> In particular, instructors introducing Python to new programmers should consider teaching Python 3 first and then introducing the differences in Python 2 afterwards (if necessary), since Python 3 [[http://python-notes.curiousefficiency.org/en/latest/python3/questions_and_answers.html#why-is-python-3-considered-a-better-language-to-teach-beginning-programmers|eliminates many quirks]] that can unnecessarily trip up beginning programmers trying to learn Python 2. |
Line 25: | Line 31: |
Firstly, if you're deploying to an environment you don't control, that may impose a specific version rather than allowing you a free selection from the available versions. | * Firstly, if you are deploying to an environment you don't control, that may require use of a specific version, rather than allowing you a free selection from the available versions. * Secondly, if you want to use a specific third party package or utility that doesn't yet have a released version that is compatible with Python 3, and porting that package is a non-trivial task, you may choose to use Python 2 in order to retain access to that package. |
Line 27: | Line 34: |
Secondly, if you want to use a specific third party package or utility that doesn't yet have a released version that is compatible with Python 3, and porting that package is a non-trivial task, you may choose to use Python 2 in order to retain access to that package. | Python 3 already broadly supports creating GUI applications, with Tkinter in the standard library. Python 3 has been supported by [[PyQt4|PyQt]] almost from the day Python 3 was released; PySide added Python 3 support in 2011. GTK+ GUIs can be created with [[https://live.gnome.org/PyGObject|PyGObject]] which supports Python 3 and is the successor to PyGtk. |
Line 29: | Line 36: |
Popular modules that don't yet support Python 3 include Twisted (for networking and a bunch of other stuff), gevent (like Twisted but different), Django and Pylons (for building websites), PyGTK and PySide (for making GUIs), py2exe (for packaging your application for Windows users), PIL (for processing images)... | Many other major packages have been ported to Python 3 including: |
Line 31: | Line 38: |
Most of these libraries have people working on 3.x support and it's mostly a work in progress in various stages of completion. For some libraries, it's more of a priority than others: Twisted, for example, is mostly focused on production servers, where supporting older versions of Python is important, let alone supporting a new version that includes major changes to the language. (Twisted is a prime example of a major package where porting to 3.x is far from trivial.) | * [[https://www.numpy.org|NumPy]] and [[https://www.scipy.org|SciPy]] (for number crunching and scientific computing) * [[https://www.djangoproject.com/|Django]], [[http://flask.pocoo.org/|Flask]], CherryPy and [[https://pyramid.readthedocs.org/en/latest|Pyramid]] (for Web sites) * PIL (an image processing module) was superseded by its fork [[https://python-pillow.org/|Pillow]], which supports Python 3. * [[http://cx-freeze.sourceforge.net/|cx_Freeze]] (for packaging applications with their dependencies) * [[http://www.py2exe.org/index.cgi/Tutorial#Step52|py2exe]] (for packaging your application for Windows users) * [[http://opencv.org|OpenCV 3]], (an open source computer vision and machine learning library) now supports Python 3 in versions 3.0 and later. * [[http://python-requests.org|Requests]], (an HTTP library for humans) * [[http://lxml.de|lxml]], (a powerful and Pythonic XML processing library combining libxml2/libxslt with the ElementTree API) * [[http://www.crummy.com/software/BeautifulSoup/bs4/|BeautifulSoup4]], (a screen-scraping library for parsing HTML and XML) * The [[http://ipython.org|IPython]]/[[http://jupyter.org|Jupyter]] project for interactive computing fully supports Python 3. * And many, many more! |
Line 33: | Line 50: |
Of course, many packages have already been ported to 3.x. For example, people who want GUI functionality could use PyQt or Tkinter (shipped with the standard library) instead of PyGTK. And for number crunching, numpy supports Python 3 since [[http://projects.scipy.org/numpy/browser/trunk/doc/release/1.5.0-notes.rst|version 1.5]]. If you want to use Python 3.x, but you're afraid to because of a dependency, it's probably worthwhile doing some research first: again, this is a work in progress and this wiki page might be outdated. | If you want to use Python 3.x, but you are afraid to because of a dependency, it's probably worthwhile doing some research first. This is a work in progress and this wiki page might be outdated (please update if you find newer information). Furthermore, with the large common subset supported by both Python 2.6+ and Python 3.3+, much modern Python code should run largely unmodified on Python 3, especially code written to interoperate with web and GUI frameworks that force applications to correctly distinguish binary data and text (some assistance from the [[https://pypi.python.org/pypi/six|six compatibility module]] may be needed to handle name changes. |
Line 35: | Line 52: |
One issue with choosing Python 3 is that a lot of documentation (including examples) on the web and in reference books will be for Python 2. This can require some adjustment to make things work with Python 3 instead. Similarly, for those inclined to seek help via IRC, the #python regulars are typically seasoned developers who rely on legacy software, most of which hasn't been ported yet (Twisted and Zope come to mind, as well as the other examples given above). As a result, they might not be able to help with Python 3.x issues or issues pertaining to libraries that have been ported to 3.x as well as they could with otherwise comparable 2.x issues. This counts for many sources of help outside of #python as well for largely the same reason. Of course, this situation is continuously improving, and some problems aren't as 3.x-specific as you might think, so it might be worthwhile to give #python or alternative sources a help a shot anyway. That said, there are still significant sources of help, such as comp.lang.python, which do have a lot of 3.x users who may be able to help you with any problems. | Even though the [[http://docs.python.org|official python documentation]] and the [[http://docs.python.org/3/tutorial/|tutorial]] have been completely updated for Python 3, there is still a lot of documentation (including examples) on the Web and in reference books that use Python 2, although more are being updated all the time. This can require some adjustment to make things work with Python 3 instead. |
Line 37: | Line 54: |
== But wouldn't I want to avoid 2.x? It's an old language with a bunch of mistakes, and it took a major version to get 'em out. == | Some people just don't want to use Python 3.x, which is their prerogative. However, they are [[2.x-vs-3.x-survey|in the minority]]. |
Line 39: | Line 56: |
Well, not really. The good news is that you don't have to drop all of the 3.x goodness because you're using 2.x. A lot of the good ideas in 3.0 were backported to 2.6, and even more of the good ideas from 3.0, 3.1 and the upcoming 3.2 will be available in 2.7. The number of things that you really can't do in 2.x but can in 3.x is pretty small: it's just not always as elegant as it is in 3.x. A non-exhaustive list of features which are only available in 3.x releases and most likely won't be backported to the 2.x series: | It is worth noting that if you wish to use an alternative implementation of Python such as IronPython, [[Jython]] or [[http://pyston.org|Pyston]] (or one of the longer list of Python platform or compiler [[PythonImplementations|implementations]]), Python 3 support is still relatively rare. This may affect you if you are interested in choosing such an implementation for reasons of integration with other systems or for performance. == But wouldn't I want to avoid 2.x? It's an old language with many mistakes, and it took a major version to get them out. == Well, not entirely. Some of the less disruptive improvements in 3.0 and 3.1 have been backported to 2.6 and 2.7, respectively. For more details on the backported features, see [[https://docs.python.org/2.6/whatsnew/2.6.html|What's New in Python 2.6]] and [[https://docs.python.org/2.7/whatsnew/2.7.html|What's New in Python 2.7]]. A non-exhaustive list of features which are only available in 3.x releases and won't be backported to the 2.x series: * strings are Unicode by default * clean Unicode/bytes separation * exception chaining |
Line 45: | Line 71: |
For more details on the backported features, see [[http://docs.python.org/release/2.6.4/whatsnew/2.6.html|What's New in Python 2.6]] and [[http://docs.python.org/dev/whatsnew/2.7.html|What's New in Python 2.7]]. | Also, language evolution is not limited to core syntactic or semantic changes. It also regards the standard library, where many improvements are done in 3.x that will not be backported directly to Python 2. See [[http://docs.python.org/3/whatsnew|What's New in Python 3]], for example. However, a number of the standard library improvements are also available through PyPI. |
Line 47: | Line 73: |
Well written 2.x code will actually be a lot like 3.x code. That can mean a lot of things, including using new-style classes, not using ancient deprecated arcane incantations of print, using lazy iterators where available... A practical example: good 2.x code will typically use xrange instead of range; xrange behaves just like 3.x' range (although range is even better in Python 3, since it can handle values larger than sys.maxint). | That said, well-written 2.x code can be a lot like 3.x code. That can mean many things, including using new-style classes, not using ancient deprecated arcane incantations of print, using lazy iterators where available, etc. A practical example: good 2.x code will typically use xrange instead of range; xrange was the starting point for the Python 3.x range implementation (although range is even better in Python 3, since it can handle values larger than sys.maxint). It should be noted that xrange() is not included in Python 3. |
Line 49: | Line 75: |
Above all, it is recommended that you focus on writing ''good'' code so that 2.x vs 3.x becomes less of an issue. That includes writing full unit test suites, and getting Unicode right (Python 3.x is significantly less forgiving than 2.x about Unicode versus bytes issues: this is generally considered to be a good thing; it just makes porting some software packages fairly annoying). | Above all, it is recommended that you focus on writing ''good'' code so that 2.x vs 3.x becomes less of an issue. That includes writing full unit test suites, and getting Unicode right. (Python 3.x is significantly less forgiving than 2.x about Unicode versus bytes issues: This is considered to be a good thing, though it makes porting some software packages fairly annoying.) |
Line 52: | Line 78: |
Well, assuming you can't find an alternative package that already supports Python 3, you still have a few options to consider: |
Assuming you can't find an alternative package that already supports Python 3, you still have a few options to consider: |
Line 59: | Line 84: |
The ideal situation is that you try to port the library to 3.x. Oftentimes you'll find someone is already working on this. Even when that's not the case, existing project members will usually appreciate the help, especially as porting often finds bugs in the original software, improving the quality of both the original and the 3.x port. Porting isn't always easy, but it's usually easier than writing your own thing from scratch. | The ideal situation is that you try to port the library to 3.x. Often you'll find someone is already working on this. Even when that's not the case, existing project members will usually appreciate the help, especially as porting often finds bugs in the original software, improving the quality of both the original and the 3.x port. Porting isn't always easy, but it's usually easier than writing your own thing from scratch. |
Line 61: | Line 86: |
How you're supposed to do porting is explained in [[http://www.python.org/dev/peps/pep-3000/#id9|PEP 3000]]. The basic idea is to take the 2.x version of the library and use the automated 2to3 converter to create a Python 3 compatible version and check that all the unit tests still pass. If tests fail, modify the original 2.x sources and try again. This approach makes it feasible to support 2.x and 3.x in parallel from a single 2.x code base. This is much easier than trying to maintain separate 2.x and 3.x branches in parallel (just ask the core Python developers about that one - they've been stuck with doing that for a couple of years now!). | How you're supposed to do porting is explained in this [[http://docs.python.org/3/howto/pyporting.html|Python 2 porting guide]]. The basic idea is to take the 2.x version of the library and check that all the unit tests still pass without warning when using the -3 command line switch in Python 2. If tests fail or emit warnings, modify the sources and try again (this may require dropping compatibility with older Python versions). Once the code runs without warnings when using the -3 switch, then try running it with Python 3. The best possible case is when this "just works" - code written using modern Python 2 idioms is source compatible with Python 3, so it's possible that the "port" may be complete at this point. |
Line 63: | Line 88: |
The porting situation is more complicated if there are C extension modules involved, but even then it is still likely to be easier than inventing your own equivalent package. | If the tests still fail under Python 3, then the standard library's 2to3 utility can often automatically create a version that will run under Python 3. Alternatively, Armin Ronacher's [[https://pypi.python.org/pypi/modernize|python-modernize]] utility instead targets the common subset of Python 2.6+ and either 3.2+ or 3.3+ (depending on the command line options used). (If using the latter, it's important to check the tests still pass under Python 2 as well!) Either approach makes it feasible to support 2.x and 3.x in parallel from a single 2.x code base. This is much easier than trying to maintain separate 2.x and 3.x branches in parallel (just ask the core Python developers about that one - they've been stuck with doing that for quite a few years now!). If the tests still fail after automated conversion or modernization, the code may be affected by a semantic change between Python 2 and 3 that the converters can't handle automatically and that isn't detected by the -3 switch. Such issues should be rare, but may still exist - if one is encountered, it's worth filing a bug against CPython requesting a new -3 warning. The porting situation is potentially more complicated if there are C extension modules involved and the project isn't using a wrapper generator like Cython, cffi or SWIG that automatically handles the differences between Python 2 and 3, but even then it is still likely to be easier than inventing your own equivalent package. The [[http://docs.python.org/3/howto/cporting.html|extension porting guide]] covers some of the key differences. |
Line 68: | Line 99: |
In addition to the 2to3 tool which allows 3.x code to be generated from 2.x source code, there's also the 3to2 tool, which aims to convert 3.x code back to 2.x code. In theory, this should work even better than going the other direction, since 3.x doesn't have as many nasty corner cases for the converter to handle (getting rid of as many of those as possible was one of the main reasons for breaking backward compatibility after all!). However, code which makes heavy use of 3.x only features (such as function annotations or extended tuple unpacking) is unlikely to be converted successfully. |
In addition to the 2to3 tool that allows 3.x code to be generated from 2.x source code, there's also the [[3to2]] tool, which aims to convert 3.x code back to 2.x code. In theory, this should work even better than going the other direction, since 3.x doesn't have as many nasty corner cases for the converter to handle (getting rid of as many of those as possible was one of the main reasons for breaking backward compatibility after all!). However, code which makes heavy use of 3.x only features (such as function annotations or extended tuple unpacking) is unlikely to be converted successfully. |
Line 73: | Line 103: |
== Supporting Python 2 and Python 3 in a common code base == The common subset of Python 2.6+ and Python 3.3+ is quite large - the restoration of u prefix support for unicode literals in Python 3.3 means that semantically correct Python 2.6+ code can be made source compatible with Python 3.3+ while still remaining largely idiomatic Python. The main difference is that some things will need to be imported from different places in order to handle the fact they have different names in Python 2 and Python 3. Accordingly, the [[http://packages.python.org/six/|six compatibility package]] is a key utility for supporting Python 2 and Python 3 in a single code base. The [[http://python-future.org/index.html|future compatibility package]] is still in beta and doesn't support as many versions of Python as six (it only goes back as far as Python 2.6, while six supports Python 2.4), but allows Python 2 compatible code to be written in a style that is closer to idiomatic Python 3 (for example, it includes an actual Python 2 compatible implementation of the Python 3 bytes type, rather than relying on the Python 2.x 8-bit string type that exposes a slightly different API). Another key thing to identify for standard library modules is if there is a more up to date backport on PyPI that can be used in preference to the 2.x standard library version. The following modules are either PyPI backports, or else the original modules that served as the source of (or inspiration for) standard library additions in Python 2.7 or 3.x: * [[https://pypi.python.org/pypi/unittest2|unittest2]] (Michael Foord, stdlib unittest maintainer, needed mostly for 2.6 support) * [[https://pypi.python.org/pypi/mock|mock]] (Michael Foord, stdlib unittest.mock maintainer) * [[https://pypi.python.org/pypi/contextlib2|contextlib2]] (Nick Coghlan, stdlib contextlib maintainer) * [[https://pypi.python.org/pypi/configparser|configparser]] (Łukasz Langa, stdlib configparser maintainer) * [[https://pypi.python.org/pypi/futures|futures]] (Alex Grönholm and Brian Quinlan, stdlib concurrent.futures maintainer) * [[https://pypi.python.org/pypi/argparse|argparse]] (Steven Bethard, stdlib argparse maintainer, needed mostly for 2.6 support) * [[https://pypi.python.org/pypi/faulthandler|faulthandler]] (Victor Stinner, stdlib faulthandler maintainer) * [[https://pypi.python.org/pypi/cdecimal/2.3|cdecimal]] (Stefan Krah, stdlib decimal maintainer) * [[https://pypi.python.org/pypi/ipaddr|ipaddr]] (Peter Moody, stdlib ipaddress maintainer, Google's IP address manipulation module that inspired the design of the stdlib ipaddress module) * [[https://pypi.python.org/pypi/stats|stats]] (Steven D'Aprano, stdlib statistics maintainer) * [[https://pypi.python.org/pypi/enum34|enum34]] (Ethan Furman, stdlib enum maintainer) * [[https://pypi.python.org/pypi/funcsigs|funcsigs]] (Aaron Iles, backport of function signature objects) * [[https://pypi.python.org/pypi/backports|shared namespace module for backports]] (Brandon Craig Rhodes) * [[https://pypi.python.org/pypi/backports.inspect|backports.inspect]] (Tripp Lilley, backport of additional inspect module changes, based on funcsigs) * [[https://pypi.python.org/pypi/backports.datetime_timestamp|backports.datetime_timestamp]] (Jason R. Coombs, backport of datetime.timestamp method as a module level function accepting a datetime object) * [[https://pypi.python.org/pypi/backports.pbkdf2|backports.pbkdf2]] (Christian Heimes, stdlib hashlib maintainer, backport of hashlib.pbkdf2_hmac) * [[https://pypi.python.org/pypi/backports.ssl_match_hostname|backports.ssl_match_hostname]] (Brandom Craig Rhodes and Toshio Kuratomi, backport of ssl.match_hostname) * [[https://pypi.python.org/pypi/backports.lzma|backports.lzma]] (Peter Cock, backport of the lzma wrapper module) * [[https://pypi.python.org/pypi/lzmaffi|lzmaffi]] (Tomer Chachamu, alternate lzma backport that uses cffi for better PyPy JIT compatibility) * [[https://pypi.python.org/pypi/tracemalloc|tracemalloc]] (Victor Stinner, stdlib tracemalloc maintainer) * [[https://pypi.python.org/pypi/pathlib|pathlib]] (Antoine Pitrou, stdlib pathlib maintainer) * [[https://pypi.python.org/pypi/selectors34|selectors34]] (Berker Peksag, backport of the stdlib selectors module) The advantage of using the backports namespace module is that it clearly indicates when something is a cross-version backport of a standard library feature, and also allows the original module name to be used when appropriate without conflicting with the standard library name. Some smaller Python 3 additions are available as recipes in the ActiveState Python Cookbook. * [[http://code.activestate.com/recipes/578078-py26-and-py30-backport-of-python-33s-lru-cache/|functools.lru_cache]] (Raymond Hettinger) The following modules aren't backports, but are cross-version compatible alternatives to key standard library APIs: * [[https://pypi.python.org/pypi/requests|requests]] (higher level HTTP and HTTPS APIs. requests itself is unlikely to be added to the stdlib for assorted technical reasons, but an equivalent client API based on asyncio is a plausible future addition) * [[https://pypi.python.org/pypi/regex|regex]] (an alternative regular expression engine with in principle approval for eventual stdlib inclusion, but requires a PEP to work out the details of the incorporation) * [[https://pypi.python.org/pypi/lxml/|lxml.etree]] (alternative implementation of the ElementTree XML API) In addition to the above modules that also support Python 2, the asyncio module added to the standard library in Python 3.4 was originally developed as a PyPI module for Python 3.3: * [[https://pypi.python.org/pypi/asyncio|asyncio]] (Guido van Rossum, BDFL and stdlib asyncio maintainer, requires "yield from" syntax added in Python 3.3) |
|
Line 74: | Line 152: |
* [[https://web.archive.org/web/20181225154158/http://getpython3.com/|Community Web site to promote Python 3]] * Nick Efford has some specific comments in relation to teaching programming with Python 3: http://www.comp.leeds.ac.uk/nde/papers/teachpy3.html * Mark Pilgrim has written a Python 3 focused version of "Dive Into Python": http://getpython3.com/diveintopython3/ * Swaroop C H has updated "A Byte of Python" to use Python 3, while keeping the last Python 2 based version available: http://www.swaroopch.com/notes/Python * "What an IronPython user should know about Python 3": http://www.itworld.com/development/104506/python-3-and-ironpython * PyCon Ireland 2010 included a talk by Paul Barry entitled "Head First into Python 3" and is available here: http://vimeo.com/groups/pyconireland/videos/14354395 - Paul has a follow-up talk from PyCon Ireland 2011 entitled "What's the scoop with Python 3?" where he talks more about the (lack of) Python 3 adoption within the community, available here: http://vimeo.com/groups/pyconireland/videos/31071871 * Mark Summerfield has written a 4 page PDF summarizing the differences between Python 2 and 3: [[http://ptgmedia.pearsoncmg.com/imprint_downloads/informit/promotions/python/python2python3.pdf|Moving from Python 2 to Python 3]] * Wesley Chun has written a couple of Python 3 articles: [[http://www.informit.com/articles/article.aspx?p=1328795|Python 3: the Evolution of a Programming Language (Mar 2009)]] and [[http://www.informit.com/articles/article.aspx?p=1439189|Python's "New" Division: Python 2 Versus Python 3 (Jan 2010)]] * Wesley Chun's [[http://us.pycon.org/2010/conference/schedule/event/29|Python 3: the Next Generation]] talk & slides (PyCon, Feb 2010) * James Bennett wrote an interesting article discussing [[http://www.b-list.org/weblog/2008/dec/05/python-3000/|why Python 3.0 exists at all]] * [[http://washort.twistedmatrix.com/2010/11/unicode-in-python-and-how-to-prevent-it.html|how to get Unicode versus bytes semantics in 2.x similar to the ones in 3.x]] (preventing implicit encoding and decoding, while keeping useful features, such as str.split/bytes.split * Nick Coghlan has [[http://python-notes.curiousefficiency.org/en/latest/python3/questions_and_answers.html|an FAQ]] cover many details of the rationale behind creating Python 3, the intended migration plan, and the current status of the transition |
|
Line 75: | Line 165: |
* Advice from the #python crew (the original inspiration for the contents of this page): http://python-commandments.org/python3.html * Nick Efford has some specific comments in relation to teaching programming with Python 3: http://www.comp.leeds.ac.uk/nde/papers/teachpy3.html * Mark Pilgrim has written a Python 3 focused version of Dive Into Python: http://diveintopython3.org/whats-new.html * "What an IronPython user should know about Python 3": http://www.itworld.com/development/104506/python-3-and-ironpython * !PyConIreland 2010 included a talk by Paul Barry entitled "Head First into Python 3" and is available here: http://vimeo.com/groups/pyconireland/videos/14354395 |
== Footnotes == |
Should I use Python 2 or Python 3 for my development activity?
You should use Python 3 going forward. As of January 2020 Python 2 will be in EOL (End Of Life) status and receive no further official support.After that date, there will be no further updates nor bugfixes. Since this end-of-life date has been planned for nearly a decade (the first end-of-life date was slated to happen in 2014, and was pushed back to 2020), and nearly all popular libraries have already ported their code, Python 2.x is well on its way to obsolescence. As such, we can only recommend learning and teaching Python 3 - which is simpler and will remain relevant well into the future.
This page is largely out of date as of October 2019, but provides a historical reference as it provides value for maintaining legacy Python 2 systems.
Contents
-
Should I use Python 2 or Python 3 for my development activity?
- What are the differences?
- Which version should I use?
- But wouldn't I want to avoid 2.x? It's an old language with many mistakes, and it took a major version to get them out.
- I want to use Python 3, but there's this tiny library I want to use that's Python 2.x only. Do I really have to revert to using Python 2 or give up on using that library?
- I decided to write something in 3.x but now someone wants to use it who only has 2.x. What do I do?
- Supporting Python 2 and Python 3 in a common code base
- Other resources that may help make the choice between Python 2 and Python 3
- Footnotes
What are the differences?
Short version: Python 2.x is legacy, Python 3.x is the present and future of the language
Python 3.0 was released in 2008. The final 2.x version 2.7 release came out in mid-2010, with a statement of extended support for this end-of-life release. The 2.x branch will see no new major releases after that. 3.x is under active development and has already seen over five years of stable releases, including version 3.3 in 2012, 3.4 in 2014, 3.5 in 2015, and 3.6 in 2016. This means that all recent standard library improvements, for example, are only available by default in Python 3.x.
Guido van Rossum (the original creator of the Python language) decided to clean up Python 2.x properly, with less regard for backwards compatibility than is the case for new releases in the 2.x range. The most drastic improvement is the better Unicode support (with all text strings being Unicode by default) as well as saner bytes/Unicode separation.
Besides, several aspects of the core language (such as print and exec being statements, integers using floor division) have been adjusted to be easier for newcomers to learn and to be more consistent with the rest of the language, and old cruft has been removed (for example, all classes are now new-style, "range()" returns a memory efficient iterable, not a list as in 2.x).
The What's New in Python 3.0 document provides a good overview of the major language changes and likely sources of incompatibility with existing Python 2.x code. Nick Coghlan (one of the CPython core developers) has also created a relatively extensive FAQ regarding the transition.
However, the broader Python ecosystem has amassed a significant amount of quality software over the years. The downside of breaking backwards compatibility in 3.x is that some of that software (especially in-house software in companies) still doesn't work on 3.x yet.
Which version should I use?
Which version you ought to use is mostly dependent on what you want to get done.
If you can do exactly what you want with Python 3.x, great! There are a few minor downsides, such as very slightly worse library support1 and the fact that some current Linux distributions and Macs still use 2.x as default, but as a language Python 3.x is definitely mature and ready for use. As long as Python 3.x is installed on your user's computers (which ought to be easy, since many people reading this may only be developing something for themselves or an environment they control) and you are writing things where you know none of the Python 2.x modules are needed, it is an excellent choice. Also, most Linux distributions have Python 3.x already installed, and nearly all have it available for end-users. One somewhat painful exception is Red Hat Enterprise Linux through version 7: Python 3 does exist in the EPEL repository, but some users may not be allowed by company policy to install anything from such add-on locations - see note on "control" below. Some distributions are phasing out Python 2 as preinstalled default.2
In particular, instructors introducing Python to new programmers should consider teaching Python 3 first and then introducing the differences in Python 2 afterwards (if necessary), since Python 3 eliminates many quirks that can unnecessarily trip up beginning programmers trying to learn Python 2.
However, there are some key issues that may require you to use Python 2 rather than Python 3.
- Firstly, if you are deploying to an environment you don't control, that may require use of a specific version, rather than allowing you a free selection from the available versions.
- Secondly, if you want to use a specific third party package or utility that doesn't yet have a released version that is compatible with Python 3, and porting that package is a non-trivial task, you may choose to use Python 2 in order to retain access to that package.
Python 3 already broadly supports creating GUI applications, with Tkinter in the standard library. Python 3 has been supported by PyQt almost from the day Python 3 was released; PySide added Python 3 support in 2011. GTK+ GUIs can be created with PyGObject which supports Python 3 and is the successor to PyGtk.
Many other major packages have been ported to Python 3 including:
NumPy and SciPy (for number crunching and scientific computing)
PIL (an image processing module) was superseded by its fork Pillow, which supports Python 3.
cx_Freeze (for packaging applications with their dependencies)
py2exe (for packaging your application for Windows users)
OpenCV 3, (an open source computer vision and machine learning library) now supports Python 3 in versions 3.0 and later.
Requests, (an HTTP library for humans)
lxml, (a powerful and Pythonic XML processing library combining libxml2/libxslt with the ElementTree API)
BeautifulSoup4, (a screen-scraping library for parsing HTML and XML)
The IPython/Jupyter project for interactive computing fully supports Python 3.
- And many, many more!
If you want to use Python 3.x, but you are afraid to because of a dependency, it's probably worthwhile doing some research first. This is a work in progress and this wiki page might be outdated (please update if you find newer information). Furthermore, with the large common subset supported by both Python 2.6+ and Python 3.3+, much modern Python code should run largely unmodified on Python 3, especially code written to interoperate with web and GUI frameworks that force applications to correctly distinguish binary data and text (some assistance from the six compatibility module may be needed to handle name changes.
Even though the official python documentation and the tutorial have been completely updated for Python 3, there is still a lot of documentation (including examples) on the Web and in reference books that use Python 2, although more are being updated all the time. This can require some adjustment to make things work with Python 3 instead.
Some people just don't want to use Python 3.x, which is their prerogative. However, they are in the minority.
It is worth noting that if you wish to use an alternative implementation of Python such as IronPython, Jython or Pyston (or one of the longer list of Python platform or compiler implementations), Python 3 support is still relatively rare. This may affect you if you are interested in choosing such an implementation for reasons of integration with other systems or for performance.
But wouldn't I want to avoid 2.x? It's an old language with many mistakes, and it took a major version to get them out.
Well, not entirely. Some of the less disruptive improvements in 3.0 and 3.1 have been backported to 2.6 and 2.7, respectively. For more details on the backported features, see What's New in Python 2.6 and What's New in Python 2.7.
A non-exhaustive list of features which are only available in 3.x releases and won't be backported to the 2.x series:
- strings are Unicode by default
- clean Unicode/bytes separation
- exception chaining
- function annotations
- syntax for keyword-only arguments
- extended tuple unpacking
- non-local variable declarations
Also, language evolution is not limited to core syntactic or semantic changes. It also regards the standard library, where many improvements are done in 3.x that will not be backported directly to Python 2. See What's New in Python 3, for example. However, a number of the standard library improvements are also available through PyPI.
That said, well-written 2.x code can be a lot like 3.x code. That can mean many things, including using new-style classes, not using ancient deprecated arcane incantations of print, using lazy iterators where available, etc. A practical example: good 2.x code will typically use xrange instead of range; xrange was the starting point for the Python 3.x range implementation (although range is even better in Python 3, since it can handle values larger than sys.maxint). It should be noted that xrange() is not included in Python 3.
Above all, it is recommended that you focus on writing good code so that 2.x vs 3.x becomes less of an issue. That includes writing full unit test suites, and getting Unicode right. (Python 3.x is significantly less forgiving than 2.x about Unicode versus bytes issues: This is considered to be a good thing, though it makes porting some software packages fairly annoying.)
I want to use Python 3, but there's this tiny library I want to use that's Python 2.x only. Do I really have to revert to using Python 2 or give up on using that library?
Assuming you can't find an alternative package that already supports Python 3, you still have a few options to consider:
- Port the library to 3.x. ("Porting" means that you make the library work on 3.x.)
- If that turns out to be really hard, and all your other dependencies do exist in 2.x, consider starting off in 2.x. As has already been explained in other places, good 2.x code will typically make switching painless as soon as every dependency has been successfully ported.
- Decide if the feature is really that important. Maybe you could drop it?
The ideal situation is that you try to port the library to 3.x. Often you'll find someone is already working on this. Even when that's not the case, existing project members will usually appreciate the help, especially as porting often finds bugs in the original software, improving the quality of both the original and the 3.x port. Porting isn't always easy, but it's usually easier than writing your own thing from scratch.
How you're supposed to do porting is explained in this Python 2 porting guide. The basic idea is to take the 2.x version of the library and check that all the unit tests still pass without warning when using the -3 command line switch in Python 2. If tests fail or emit warnings, modify the sources and try again (this may require dropping compatibility with older Python versions). Once the code runs without warnings when using the -3 switch, then try running it with Python 3. The best possible case is when this "just works" - code written using modern Python 2 idioms is source compatible with Python 3, so it's possible that the "port" may be complete at this point.
If the tests still fail under Python 3, then the standard library's 2to3 utility can often automatically create a version that will run under Python 3. Alternatively, Armin Ronacher's python-modernize utility instead targets the common subset of Python 2.6+ and either 3.2+ or 3.3+ (depending on the command line options used). (If using the latter, it's important to check the tests still pass under Python 2 as well!)
Either approach makes it feasible to support 2.x and 3.x in parallel from a single 2.x code base. This is much easier than trying to maintain separate 2.x and 3.x branches in parallel (just ask the core Python developers about that one - they've been stuck with doing that for quite a few years now!).
If the tests still fail after automated conversion or modernization, the code may be affected by a semantic change between Python 2 and 3 that the converters can't handle automatically and that isn't detected by the -3 switch. Such issues should be rare, but may still exist - if one is encountered, it's worth filing a bug against CPython requesting a new -3 warning.
The porting situation is potentially more complicated if there are C extension modules involved and the project isn't using a wrapper generator like Cython, cffi or SWIG that automatically handles the differences between Python 2 and 3, but even then it is still likely to be easier than inventing your own equivalent package. The extension porting guide covers some of the key differences.
There are also some more in depth guides right here on the wiki: PortingPythonToPy3k, PortingExtensionModulesToPy3k
I decided to write something in 3.x but now someone wants to use it who only has 2.x. What do I do?
In addition to the 2to3 tool that allows 3.x code to be generated from 2.x source code, there's also the 3to2 tool, which aims to convert 3.x code back to 2.x code. In theory, this should work even better than going the other direction, since 3.x doesn't have as many nasty corner cases for the converter to handle (getting rid of as many of those as possible was one of the main reasons for breaking backward compatibility after all!). However, code which makes heavy use of 3.x only features (such as function annotations or extended tuple unpacking) is unlikely to be converted successfully.
It's probably also fair to say that 3to2 is the road less traveled compared to 2to3 at this stage, so you might come across a few rough edges here and there. However, if you want to write 3.x code, it's definitely an idea worth exploring.
Supporting Python 2 and Python 3 in a common code base
The common subset of Python 2.6+ and Python 3.3+ is quite large - the restoration of u prefix support for unicode literals in Python 3.3 means that semantically correct Python 2.6+ code can be made source compatible with Python 3.3+ while still remaining largely idiomatic Python. The main difference is that some things will need to be imported from different places in order to handle the fact they have different names in Python 2 and Python 3.
Accordingly, the six compatibility package is a key utility for supporting Python 2 and Python 3 in a single code base.
The future compatibility package is still in beta and doesn't support as many versions of Python as six (it only goes back as far as Python 2.6, while six supports Python 2.4), but allows Python 2 compatible code to be written in a style that is closer to idiomatic Python 3 (for example, it includes an actual Python 2 compatible implementation of the Python 3 bytes type, rather than relying on the Python 2.x 8-bit string type that exposes a slightly different API).
Another key thing to identify for standard library modules is if there is a more up to date backport on PyPI that can be used in preference to the 2.x standard library version. The following modules are either PyPI backports, or else the original modules that served as the source of (or inspiration for) standard library additions in Python 2.7 or 3.x:
unittest2 (Michael Foord, stdlib unittest maintainer, needed mostly for 2.6 support)
mock (Michael Foord, stdlib unittest.mock maintainer)
contextlib2 (Nick Coghlan, stdlib contextlib maintainer)
configparser (Łukasz Langa, stdlib configparser maintainer)
futures (Alex Grönholm and Brian Quinlan, stdlib concurrent.futures maintainer)
argparse (Steven Bethard, stdlib argparse maintainer, needed mostly for 2.6 support)
faulthandler (Victor Stinner, stdlib faulthandler maintainer)
cdecimal (Stefan Krah, stdlib decimal maintainer)
ipaddr (Peter Moody, stdlib ipaddress maintainer, Google's IP address manipulation module that inspired the design of the stdlib ipaddress module)
stats (Steven D'Aprano, stdlib statistics maintainer)
enum34 (Ethan Furman, stdlib enum maintainer)
funcsigs (Aaron Iles, backport of function signature objects)
shared namespace module for backports (Brandon Craig Rhodes)
backports.inspect (Tripp Lilley, backport of additional inspect module changes, based on funcsigs)
backports.datetime_timestamp (Jason R. Coombs, backport of datetime.timestamp method as a module level function accepting a datetime object)
backports.pbkdf2 (Christian Heimes, stdlib hashlib maintainer, backport of hashlib.pbkdf2_hmac)
backports.ssl_match_hostname (Brandom Craig Rhodes and Toshio Kuratomi, backport of ssl.match_hostname)
backports.lzma (Peter Cock, backport of the lzma wrapper module)
lzmaffi (Tomer Chachamu, alternate lzma backport that uses cffi for better PyPy JIT compatibility)
tracemalloc (Victor Stinner, stdlib tracemalloc maintainer)
pathlib (Antoine Pitrou, stdlib pathlib maintainer)
selectors34 (Berker Peksag, backport of the stdlib selectors module)
The advantage of using the backports namespace module is that it clearly indicates when something is a cross-version backport of a standard library feature, and also allows the original module name to be used when appropriate without conflicting with the standard library name.
Some smaller Python 3 additions are available as recipes in the ActiveState Python Cookbook.
* functools.lru_cache (Raymond Hettinger)
The following modules aren't backports, but are cross-version compatible alternatives to key standard library APIs:
requests (higher level HTTP and HTTPS APIs. requests itself is unlikely to be added to the stdlib for assorted technical reasons, but an equivalent client API based on asyncio is a plausible future addition)
regex (an alternative regular expression engine with in principle approval for eventual stdlib inclusion, but requires a PEP to work out the details of the incorporation)
lxml.etree (alternative implementation of the ElementTree XML API)
In addition to the above modules that also support Python 2, the asyncio module added to the standard library in Python 3.4 was originally developed as a PyPI module for Python 3.3:
asyncio (Guido van Rossum, BDFL and stdlib asyncio maintainer, requires "yield from" syntax added in Python 3.3)
Other resources that may help make the choice between Python 2 and Python 3
Nick Efford has some specific comments in relation to teaching programming with Python 3: http://www.comp.leeds.ac.uk/nde/papers/teachpy3.html
Mark Pilgrim has written a Python 3 focused version of "Dive Into Python": http://getpython3.com/diveintopython3/
Swaroop C H has updated "A Byte of Python" to use Python 3, while keeping the last Python 2 based version available: http://www.swaroopch.com/notes/Python
"What an IronPython user should know about Python 3": http://www.itworld.com/development/104506/python-3-and-ironpython
PyCon Ireland 2010 included a talk by Paul Barry entitled "Head First into Python 3" and is available here: http://vimeo.com/groups/pyconireland/videos/14354395 - Paul has a follow-up talk from PyCon Ireland 2011 entitled "What's the scoop with Python 3?" where he talks more about the (lack of) Python 3 adoption within the community, available here: http://vimeo.com/groups/pyconireland/videos/31071871
Mark Summerfield has written a 4 page PDF summarizing the differences between Python 2 and 3: Moving from Python 2 to Python 3
Wesley Chun has written a couple of Python 3 articles: Python 3: the Evolution of a Programming Language (Mar 2009) and Python's "New" Division: Python 2 Versus Python 3 (Jan 2010)
Wesley Chun's Python 3: the Next Generation talk & slides (PyCon, Feb 2010)
James Bennett wrote an interesting article discussing why Python 3.0 exists at all
how to get Unicode versus bytes semantics in 2.x similar to the ones in 3.x (preventing implicit encoding and decoding, while keeping useful features, such as str.split/bytes.split
Nick Coghlan has an FAQ cover many details of the rationale behind creating Python 3, the intended migration plan, and the current status of the transition
Footnotes
Amongst still-maintained packages: https://python3wos.appspot.com (1)
Arch Linux links python to python3, and Ubuntu and Fedora switch defaults: https://wiki.ubuntu.com/Python/3 https://fedoraproject.org/wiki/Changes/Python_3_as_Default (2)