Porting from Python 3.4 to 3.5

Some small incompatibilities have been noticed between Python 3.4 and 3.5. This page collects the known differences and where possible, describes how to mitigate them, along with links to tracker issues where available. They are listed in no particular order.

ast API changes

The ast modules API is not guaranteed to be stable between Python releases, and indeed it has changed in Python 3.5. This breaks several projects which haven't yet ported to the new API, including genshi and pytest.

A workaround for the latter is to invoke pytest with --assert=plain.

The %x string format

The %x string formatter, as well as a few others, accepted non-integer values in earlier Python 3 versions. In Python 3.4 this was changed to issue a DeprecationWarning, although usually silenced. In Python 3.5 this was changed to raise a TypeError. For example:

>>> from uuid import uuid4
>>> '%.32x' % uuid4()

In Python 3.4, this prints a string, but in Python 3.5 you get a TypeError. In this specific case, a fix compatible with both Python versions is:

>>> '%.32x' % uuid4().int

but YMMV depending on exactly what type of object you had been passing to the format string.

Nested classes use __qualname__

In Python 3.4, when a class was printed, for example when an exception was raised in a doctest, the class was printed using '%s.%s' % (self.__class__.__module__, self.__class__.__name__) but this was inaccurate when the class was nested. In Python 3.5 this was changed to use the class's __qualname__ attribute. While more accurate, this can break doctests, as seen in this zope.testing bug.

For something like a doctest, there unfortunately is no way to write a compatible doctest for both Python 3.4 and 3.5 without writing a custom checker for doctest.DocTestSuite(), which is how zope.testing was fixed. Or of course, you could not raise a nested class.

smtplib.login()

In Python 3.5, smtplib.login() was enhanced to allow for extensible authorization hooks, and a new smtplib.auth() method was added, which .login() uses. There is a subtle behavioral difference exposed by this change though. RFC 4954 allows for the SMTP AUTH command to include an optional initial-response which eliminates a round trip for auth methods that don't require a challenge, such as AUTH PLAIN. Because handling a challenge-response is trickier for SMTP servers, some may not do the right thing when presented with a challenge-response instead of an initial-response.

The resolution of this for compatibility with Python 3.4 and 3.5 is two-fold. First, we are working in issue 15014 to restore the initial-response behavior in smtplib for auth methods that can support it. Stay tuned; this will hopefully appear in 3.5 final. For handling challenge-responses in smtpd-based servers, see issue 21935, although this is a bit more controversial. You might also be interested in this commit to GNU Mailman 3 which kludges in handling of the challenge-response in its localhost test server.

email.__version__

In Python 3.5, email.__version__ was removed.

html.parser.HTMLParseError

This exception has been removed, and other changes to html.parser module since 3.4. This affected Sphinx which imported the exception through six.moves.html_parser.

Resolution: don't import this exception. It was deprecated anyway since Python 3.3

PEP 479

Changes the behavior when StopIteration is raised inside an iterator, and clarifies that exhausted iterators are supposed to return instead of raising that exception. We've seen some failures when the deprecation warnings that raising the exception in Python 3.5 causes, produce unexpected stderr output.

Resolution: Don't raise StopIteration when your iterator is exhausted, just 'return'. This will work in all Pythons supporting iterators.

.assert_called() on mock objects

For some reason, I've seen quite a few test suites that call the .assert_called() method on a mock object, thinking that this will prove that the mock was called, irrespective of any arguments. Actually, this does nothing! Typically, mock objects just create the nonexistent attributes on the fly, but in Python 3.5, mocks have been changed to raise AttributeError when the missing method starts with assert (or its common misspelling assret). This means that mock.assert_called() at best did nothing and at worst masked a failing test in older Pythons, so it's good that Python 3.5 catches these.

Use this instead:

    self.assertTrue(mock_object.called)

PortingToPy3k/34to35 (last edited 2015-08-20 21:14:33 by BarryWarsaw)

Unable to edit the page? See the FrontPage for instructions.