Differences between revisions 1 and 51 (spanning 50 versions)
Revision 1 as of 2010-06-21 11:48:48
Size: 5829
Editor: NickCoghlan
Comment:
Revision 51 as of 2011-12-09 16:59:37
Size: 13234
Comment: Add PyGObject URL
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
#acl Known:read,write,revert All:read
Line 2: Line 4:

''''NOTE: This page is a work in progress, based on the version the #python folks put together for their own needs (see link at the end). Editors feel free to drop this notice once the content has matured a bit''''
Line 7: Line 7:
''Short version: Python 2.x is the status quo, Python 3.x is the shiny new thing.'' ''Short version: Python 2.x is the status quo, Python 3.x is the present and future of the language''
Line 9: Line 9:
At the time of writing (20 Jun 2010), both are still under development. There's at least one more 2.x release planned (2.7), but the 2.x branch probably won't see any new major releases after that. 3.x is under active and continued development. At the time of writing (July 4, 2010), the final 2.7 release is out, with a statement of extended support for this end-of-life release. The 2.x branch will see no new major releases after that. 3.x is under active development and has already seen stable releases, such as the recent 3.2. This means that all recent standard library improvements, for example, are only available in Python 3.x.
Line 11: Line 11:
3.x is the newest branch of Python. People decided to clean up Python 2.x properly, with less regard for backwards compatibility than new releases in the 2.x range. Nick Efford has a nice writeup about why you might like Python 3.x as a language here: http://www.comp.leeds.ac.uk/nde/papers/teachpy3.html. Guido van Rossum (the original creator of the Python language) decided to clean up Python 2.x properly, with less regard for backwards compatibility than is the case for new releases in the 2.x range. The most drastic improvement is the better unicode support (with all text strings being unicode by default) as well as saner bytes/unicode separation.
Line 13: Line 13:
Python has amassed a pretty big amount of quality software over the years. The downside of breaking backwards compatibility in 3.x is that a lot of that software doesn't work on 3.x (yet). Besides, several aspects of the core language (such as print and exec being statements, integers using floor division) have been adjusted to be easier for newcomers to learn and to be more consistent with the rest of the language, and old cruft has been removed (for example, all classes are now new-style, `range()` returns a memory efficient iterable, not a list as in 2.x).
Line 15: Line 15:
== So what should I pick? == The [[http://docs.python.org/py3k/whatsnew/3.0.html|What's New in Python 3.0]] document provides a good overview of the major language changes and likely sources of incompatibility with existing Python 2.x code.
Line 17: Line 17:
What you ought to pick is mostly dependant on what you want to get done. However, the broader Python ecosystem has amassed a significant amount of quality software over the years. The downside of breaking backwards compatibility in 3.x is that a lot of that software doesn't work on 3.x yet.
Line 19: Line 19:
If you can do exactly what you want with Python 3.x, great! There's a few downsides, such as library support and current Linux distributions and Macs still shipping with 2.x by default, but as a language Python 3.x is definitely ready. As long as actually getting Python 3.x on your user's computers (which ought to be easy since a lot of people reading this might only be developing something for themselves) and you're writing things where lack of third party software isn't a major impediment, Python 3.x is an excellent choice. == So which version should I use? ==
Line 21: Line 21:
Most of the current #python regulars will recommend Python 2.x because a lot of libraries that make your life easier only work on 2.x so far. Popular ones include Twisted (for networking and a bunch of other stuff), gevent (like Twisted but different), Django (for building websites), PyGTK (for making GUIs), py2exe (for packaging your application for Windows users), PIL (for processing images), numpy (for number crunching)... Which version you ought to use is mostly dependent on what you want to get done.
Line 23: Line 23:
Most of these libraries have people working on 3.x support and it's mostly a work in progress in various stages of completion. numpy, for example, is pretty close to completion. For some libraries, it's more of a priority than others: Twisted, for example, is mostly focused on production servers, where people are often using old versions of 2.x, let alone thinking about 3.x. If you can do exactly what you want with Python 3.x, great! There's a few downsides, such as comparatively limited library support and the fact that most current Linux distributions and Macs are still using 2.x as default, but as a language Python 3.x is definitely ready. As long as actually getting Python 3.x on your user's computers (which ought to be easy since a lot of people reading this may only be developing something for themselves or an environment they control) and you're writing things where lack of third party software isn't a major impediment (or where you know the packages you need already support Python 3), Python 3.x is an excellent choice. Also, several distributions have Python 3.x already installed or available for end-users.
Line 25: Line 25:
One of the problems with picking 3.x stuff is that most of the regulars are professional 2.x developers, so we might not be able to help as effectively with questions about new 3.x software as opposed to exisitng 2.x software such as Twisted, PIL or PyGTK. However, there are some key issues that may require you to use Python 2 rather than Python 3.
Line 27: Line 27:
== So wouldn't I want to avoid 2.x? It's an old language with a bunch of mistakes, and it took a major version to get 'em out. == Firstly, if you're deploying to an environment you don't control, that may impose a specific version rather than allowing you a free selection from the available versions.
Line 29: Line 29:
Well, not really. The good news is that you don't have to drop all of the 3.x goodness because you're using 2.x. A lot of the good ideas in 3.x have been backported to 2.x. The things that you really can't do in 2.x but can in 3.x is pretty small: it's just not always as elegant as it is in 3.x. Examples include an ordered dict (which might be coming to 2.7), and the "nonlocal" keyword, which probably won't be coming to 2.x (but you can work around that, and it's not really a feature you'll need every day). You might start seeing a few libraries here and there that use 3.x, but most of the time they'll have 2.x versions too. Secondly, if you want to use a specific third party package or utility that doesn't yet have a released version that is compatible with Python 3, and porting that package is a non-trivial task, you may choose to use Python 2 in order to retain access to that package.
Line 31: Line 31:
Good 2.x code will behave a lot like new 3.x code. That can mean a lot of things, including using new-style classes, not using ancient deprecated arcane incantations of print, lazy iterators where available... Popular modules that don't yet support Python 3 include Twisted (for networking and a bunch of other stuff), gevent (like Twisted but different), Django and Pylons (for building websites), py2exe (for packaging your application for Windows users), and PIL (for processing images).
Line 33: Line 33:
#python will always above all recommend that you focus on writing *good* code so that 2.x vs 3.x becomes less of an issue. That includes writing full unit test suites, and getting unicode right (Python 3.x is less forgiving about unicode versus bytes: this is generally considered to be a very good thing; it just makes porting software that doesn't make the distinction very well annoying). Most of these libraries have people working on 3.x support and it's mostly a work in progress in various stages of completion. For some libraries, it's more of a priority than others: Twisted, for example, is mostly focused on production servers, where supporting older versions of Python is important, let alone supporting a new version that includes major changes to the language. (Twisted is a prime example of a major package where porting to 3.x is far from trivial.)
Line 35: Line 35:
== I want to use Python 3, but there's this tiny library I want to use that's Python 2.x only. Do I really have to? == For creating GUI applications Python 3 already comes with Tkinter, and has been supported by PyQt4 almost from the day Python 3 was released; PySide added Python 3 support in 2011. GTK+ GUIs can be created with [[https://live.gnome.org/PyGObject|PyGObject]] which supports Python 3 and is the successor to PyGtk.
Line 37: Line 37:
First things first, and I can't stress this enough: please don't start implementing your own thing. This is called NIH (Not Invented Here) syndrome and it has produced a lot of bad software. Please, please, go help an existing project if it exists, they'll be glad to see people helping, they can assist you with questions, and your work benefits the entire Python community as a whole. Many other major packages have been ported to Python 3 including numpy (for number crunching), CherryPy (for websites) and cx_Freeze (for packaging applications with their dependencies). If you want to use Python 3.x, but you're afraid to because of a dependency, it's probably worthwhile doing some research first: again, this is a work in progress and this wiki page might be outdated.
Line 39: Line 39:
Anyway, you don't have to use Python 2.x. You could also make the Python 3.x vs 2.x situation better by porting an existing library and making future users happy. Porting isn't always easy, but it's usually easier than writing your own thing from scratch. Existing project members will love you, because porting often finds bugs in the original software, improving the quality of both the original and the 3.x port. Even if the [[http://docs.python.org/py3k/|official python documentation]] and the [[http://docs.python.org/py3k/tutorial/|tutorial]] have been updated for Python 3, there is still a lot of documentation (including examples) on the web and in reference books that still use Python 2. This can require some adjustment to make things work with Python 3 instead.
Line 41: Line 41:
How you're supposed to do porting is explained in PEP3000: http://www.python.org/dev/peps/pep-3000/#id9. It is worth noting that if you wish to use an alternative implementation of Python such as IronPython, [[Jython]] or PyPy (or one of the longer list of Python platform or compiler [[PythonImplementations|implementations]]), Python 3 support is still relatively rare. This may affect you if you are interested in choosing such an implementation for reasons of integration with other systems or for performance.

See the [[Python3PortingStatus|Python 3 Porting Status]] page for information about which libraries have been, or are in the process of being, ported to Python 3.

== But wouldn't I want to avoid 2.x? It's an old language with a bunch of mistakes, and it took a major version to get 'em out. ==

Well, not entirely. Some of the less disruptive improvements in 3.0 and 3.1 have been backported to 2.6 and 2.7, respectively. For more details on the backported features, see [[http://docs.python.org/release/2.6.4/whatsnew/2.6.html|What's New in Python 2.6]] and [[http://docs.python.org/dev/whatsnew/2.7.html|What's New in Python 2.7]].

A non-exhaustive list of features which are only available in 3.x releases and won't be backported to the 2.x series:
 * strings are unicode by default
 * clean unicode/bytes separation
 * exception chaining
 * function annotations
 * syntax for keyword-only arguments
 * extended tuple unpacking
 * non-local variable declarations

Also, language evolution is not limited to core syntactic or semantic changes. It also regards the standard library, where many improvements are done in 3.x that will not be backported to Python 2. See [[http://docs.python.org/py3k/whatsnew/3.2.html|What's New in Python 3.2]] for example.

That said, well-written 2.x code can be a lot like 3.x code. That can mean a lot of things, including using new-style classes, not using ancient deprecated arcane incantations of print, using lazy iterators where available... A practical example: good 2.x code will typically use xrange instead of range; xrange was the starting point for the Python 3.x' range implementation (although range is even better in Python 3, since it can handle values larger than sys.maxint).

Above all, it is recommended that you focus on writing ''good'' code so that 2.x vs 3.x becomes less of an issue. That includes writing full unit test suites, and getting Unicode right (Python 3.x is significantly less forgiving than 2.x about Unicode versus bytes issues: this is considered to be a good thing; it just makes porting some software packages fairly annoying).

== I want to use Python 3, but there's this tiny library I want to use that's Python 2.x only. Do I really have to revert to using Python 2 or give up on using that library? ==

Well, assuming you can't find an alternative package that already supports Python 3, you still have a few options to consider:

 * Port the library to 3.x. ("Porting" means that you make the library work on 3.x.)
 * It is possible that the library isn't really needed and you can use another one.
 * If that turns out to be really hard, and all your other dependencies do exist in 2.x, consider starting off in 2.x. As has already been explained in other places, good 2.x code will typically make switching painless as soon as every dependency has been successfully ported.
 * Decide if the feature is really that important. Maybe you could drop it?

The ideal situation is that you try to port the library to 3.x. Oftentimes you'll find someone is already working on this. Even when that's not the case, existing project members will usually appreciate the help, especially as porting often finds bugs in the original software, improving the quality of both the original and the 3.x port. Porting isn't always easy, but it's usually easier than writing your own thing from scratch.

How you're supposed to do porting is explained in [[http://www.python.org/dev/peps/pep-3000/#id9|PEP 3000]]. The basic idea is to take the 2.x version of the library and use the automated 2to3 converter to create a Python 3 compatible version and check that all the unit tests still pass. If tests fail, modify the original 2.x sources and try again. This approach makes it feasible to support 2.x and 3.x in parallel from a single 2.x code base. This is much easier than trying to maintain separate 2.x and 3.x branches in parallel (just ask the core Python developers about that one - they've been stuck with doing that for a couple of years now!).

The porting situation is more complicated if there are C extension modules involved, but even then it is still likely to be easier than inventing your own equivalent package.

There are also some more in depth guides right here on the wiki: PortingPythonToPy3k, PortingExtensionModulesToPy3k
Line 45: Line 83:
There's also the 3to2 tool, which tries it's best to convert 3.x code back to 2.x code. In addition to the 2to3 tool which allows 3.x code to be generated from 2.x source code, there's also the [[3to2]] tool, which aims to convert 3.x code back to 2.x code. In theory, this should work even better than going the other direction, since 3.x doesn't have as many nasty corner cases for the converter to handle (getting rid of as many of those as possible was one of the main reasons for breaking backward compatibility after all!). However, code which makes heavy use of 3.x only features (such as function annotations or extended tuple unpacking) is unlikely to be converted successfully.
Line 47: Line 85:
== Other Resources Discussing the Pros and Cons of Python 2 and Python 3 == It's probably also fair to say that 3to2 is the road less traveled compared to 2to3 at this stage, so you might come across a few rough edges here and there. However, if you want to write 3.x code, it's definitely an idea worth exploring.
Line 49: Line 87:
Advice from the #python crew (also the original inspiration for the contents of this page): http://python-commandments.org/python3.html
Specific comments in relation to teaching programming with Python: http://www.comp.leeds.ac.uk/nde/papers/teachpy3.html
[[http://packages.python.org/six/|six]] is another tool.

== Other resources that may help make the choice between Python 2 and Python 3 ==

 * [[http://getpython3.com/|Community website to promote Python 3]]
 * Advice from the #python crew (the original inspiration for the contents of this page): http://python-commandments.org/python3.html
 * Nick Efford has some specific comments in relation to teaching programming with Python 3: http://www.comp.leeds.ac.uk/nde/papers/teachpy3.html
 * Mark Pilgrim has written a Python 3 focused version of "Dive Into Python": http://diveintopython3.org/whats-new.html
 * Swaroop C H has updated "A Byte of Python" to use Python 3, while keeping the last Python 2 based version available: http://www.swaroopch.com/notes/Python
 * "What an IronPython user should know about Python 3": http://www.itworld.com/development/104506/python-3-and-ironpython
 * !PyCon Ireland 2010 included a talk by Paul Barry entitled "Head First into Python 3" and is available here: http://vimeo.com/groups/pyconireland/videos/14354395 - Paul has a follow-up talk from PyCon Ireland 2011 entitled "What's the scoop with Python 3?" where he talks more about the (lack of) Python 3 adoption within the community, available here: http://vimeo.com/groups/pyconireland/videos/31071871
 * Mark Summerfield has written a 4 page PDF summarizing the differences between Python 2 and 3: [[http://ptgmedia.pearsoncmg.com/imprint_downloads/informit/promotions/python/python2python3.pdf|Moving from Python 2 to Python 3]]
 * Wesley Chun has written a couple of Python 3 articles: [[http://www.informit.com/articles/article.aspx?p=1328795|Python 3: the Evolution of a Programming Language (Mar 2009)]] and [[http://www.informit.com/articles/article.aspx?p=1439189|Python's "New" Division: Python 2 Versus Python 3 (Jan 2010)]]
 * Wesley Chun's [[http://us.pycon.org/2010/conference/schedule/event/29|Python 3: the Next Generation]] talk & slides (PyCon, Feb 2010)
 * James Bennett wrote an interesting article discussing [[http://www.b-list.org/weblog/2008/dec/05/python-3000/ | why Python 3.0 exists at all]]
 * [[http://washort.twistedmatrix.com/2010/11/unicode-in-python-and-how-to-prevent-it.html|how to get Unicode versus bytes semantics in 2.x similar to the ones in 3.x]] (preventing implicit encoding and decoding, while keeping useful features such as str.split/bytes.split

== Footnotes ==
<<FootNote()>>

Should I use Python 2 or Python 3 for my development activity?

What are the differences?

Short version: Python 2.x is the status quo, Python 3.x is the present and future of the language

At the time of writing (July 4, 2010), the final 2.7 release is out, with a statement of extended support for this end-of-life release. The 2.x branch will see no new major releases after that. 3.x is under active development and has already seen stable releases, such as the recent 3.2. This means that all recent standard library improvements, for example, are only available in Python 3.x.

Guido van Rossum (the original creator of the Python language) decided to clean up Python 2.x properly, with less regard for backwards compatibility than is the case for new releases in the 2.x range. The most drastic improvement is the better unicode support (with all text strings being unicode by default) as well as saner bytes/unicode separation.

Besides, several aspects of the core language (such as print and exec being statements, integers using floor division) have been adjusted to be easier for newcomers to learn and to be more consistent with the rest of the language, and old cruft has been removed (for example, all classes are now new-style, range() returns a memory efficient iterable, not a list as in 2.x).

The What's New in Python 3.0 document provides a good overview of the major language changes and likely sources of incompatibility with existing Python 2.x code.

However, the broader Python ecosystem has amassed a significant amount of quality software over the years. The downside of breaking backwards compatibility in 3.x is that a lot of that software doesn't work on 3.x yet.

So which version should I use?

Which version you ought to use is mostly dependent on what you want to get done.

If you can do exactly what you want with Python 3.x, great! There's a few downsides, such as comparatively limited library support and the fact that most current Linux distributions and Macs are still using 2.x as default, but as a language Python 3.x is definitely ready. As long as actually getting Python 3.x on your user's computers (which ought to be easy since a lot of people reading this may only be developing something for themselves or an environment they control) and you're writing things where lack of third party software isn't a major impediment (or where you know the packages you need already support Python 3), Python 3.x is an excellent choice. Also, several distributions have Python 3.x already installed or available for end-users.

However, there are some key issues that may require you to use Python 2 rather than Python 3.

Firstly, if you're deploying to an environment you don't control, that may impose a specific version rather than allowing you a free selection from the available versions.

Secondly, if you want to use a specific third party package or utility that doesn't yet have a released version that is compatible with Python 3, and porting that package is a non-trivial task, you may choose to use Python 2 in order to retain access to that package.

Popular modules that don't yet support Python 3 include Twisted (for networking and a bunch of other stuff), gevent (like Twisted but different), Django and Pylons (for building websites), py2exe (for packaging your application for Windows users), and PIL (for processing images).

Most of these libraries have people working on 3.x support and it's mostly a work in progress in various stages of completion. For some libraries, it's more of a priority than others: Twisted, for example, is mostly focused on production servers, where supporting older versions of Python is important, let alone supporting a new version that includes major changes to the language. (Twisted is a prime example of a major package where porting to 3.x is far from trivial.)

For creating GUI applications Python 3 already comes with Tkinter, and has been supported by PyQt4 almost from the day Python 3 was released; PySide added Python 3 support in 2011. GTK+ GUIs can be created with PyGObject which supports Python 3 and is the successor to PyGtk.

Many other major packages have been ported to Python 3 including numpy (for number crunching), CherryPy (for websites) and cx_Freeze (for packaging applications with their dependencies). If you want to use Python 3.x, but you're afraid to because of a dependency, it's probably worthwhile doing some research first: again, this is a work in progress and this wiki page might be outdated.

Even if the official python documentation and the tutorial have been updated for Python 3, there is still a lot of documentation (including examples) on the web and in reference books that still use Python 2. This can require some adjustment to make things work with Python 3 instead.

It is worth noting that if you wish to use an alternative implementation of Python such as IronPython, Jython or PyPy (or one of the longer list of Python platform or compiler implementations), Python 3 support is still relatively rare. This may affect you if you are interested in choosing such an implementation for reasons of integration with other systems or for performance.

See the Python 3 Porting Status page for information about which libraries have been, or are in the process of being, ported to Python 3.

But wouldn't I want to avoid 2.x? It's an old language with a bunch of mistakes, and it took a major version to get 'em out.

Well, not entirely. Some of the less disruptive improvements in 3.0 and 3.1 have been backported to 2.6 and 2.7, respectively. For more details on the backported features, see What's New in Python 2.6 and What's New in Python 2.7.

A non-exhaustive list of features which are only available in 3.x releases and won't be backported to the 2.x series:

  • strings are unicode by default
  • clean unicode/bytes separation
  • exception chaining
  • function annotations
  • syntax for keyword-only arguments
  • extended tuple unpacking
  • non-local variable declarations

Also, language evolution is not limited to core syntactic or semantic changes. It also regards the standard library, where many improvements are done in 3.x that will not be backported to Python 2. See What's New in Python 3.2 for example.

That said, well-written 2.x code can be a lot like 3.x code. That can mean a lot of things, including using new-style classes, not using ancient deprecated arcane incantations of print, using lazy iterators where available... A practical example: good 2.x code will typically use xrange instead of range; xrange was the starting point for the Python 3.x' range implementation (although range is even better in Python 3, since it can handle values larger than sys.maxint).

Above all, it is recommended that you focus on writing good code so that 2.x vs 3.x becomes less of an issue. That includes writing full unit test suites, and getting Unicode right (Python 3.x is significantly less forgiving than 2.x about Unicode versus bytes issues: this is considered to be a good thing; it just makes porting some software packages fairly annoying).

I want to use Python 3, but there's this tiny library I want to use that's Python 2.x only. Do I really have to revert to using Python 2 or give up on using that library?

Well, assuming you can't find an alternative package that already supports Python 3, you still have a few options to consider:

  • Port the library to 3.x. ("Porting" means that you make the library work on 3.x.)
  • It is possible that the library isn't really needed and you can use another one.
  • If that turns out to be really hard, and all your other dependencies do exist in 2.x, consider starting off in 2.x. As has already been explained in other places, good 2.x code will typically make switching painless as soon as every dependency has been successfully ported.
  • Decide if the feature is really that important. Maybe you could drop it?

The ideal situation is that you try to port the library to 3.x. Oftentimes you'll find someone is already working on this. Even when that's not the case, existing project members will usually appreciate the help, especially as porting often finds bugs in the original software, improving the quality of both the original and the 3.x port. Porting isn't always easy, but it's usually easier than writing your own thing from scratch.

How you're supposed to do porting is explained in PEP 3000. The basic idea is to take the 2.x version of the library and use the automated 2to3 converter to create a Python 3 compatible version and check that all the unit tests still pass. If tests fail, modify the original 2.x sources and try again. This approach makes it feasible to support 2.x and 3.x in parallel from a single 2.x code base. This is much easier than trying to maintain separate 2.x and 3.x branches in parallel (just ask the core Python developers about that one - they've been stuck with doing that for a couple of years now!).

The porting situation is more complicated if there are C extension modules involved, but even then it is still likely to be easier than inventing your own equivalent package.

There are also some more in depth guides right here on the wiki: PortingPythonToPy3k, PortingExtensionModulesToPy3k

I decided to write something in 3.x but now someone wants to use it who only has 2.x. What do I do?

In addition to the 2to3 tool which allows 3.x code to be generated from 2.x source code, there's also the 3to2 tool, which aims to convert 3.x code back to 2.x code. In theory, this should work even better than going the other direction, since 3.x doesn't have as many nasty corner cases for the converter to handle (getting rid of as many of those as possible was one of the main reasons for breaking backward compatibility after all!). However, code which makes heavy use of 3.x only features (such as function annotations or extended tuple unpacking) is unlikely to be converted successfully.

It's probably also fair to say that 3to2 is the road less traveled compared to 2to3 at this stage, so you might come across a few rough edges here and there. However, if you want to write 3.x code, it's definitely an idea worth exploring.

six is another tool.

Other resources that may help make the choice between Python 2 and Python 3

Footnotes

Python2orPython3 (last edited 2020-06-17 20:07:07 by MatsWichmann)

Unable to edit the page? See the FrontPage for instructions.