List of tests that are failing in the py3k branch

We're now maintaining a list of tasks in a shared spreadsheet: http://spreadsheets.google.com/ccc?key=pBLWM8elhFAmKbrhhh0ApQA

Protocol:

if you start working on a test, to avoid duplicate work, add your name+email+date to the bulleted item for the test below, with the text "MINE"
if you figure out why a test is failing, just add a note (name etc. is still appreciated)
if you fix a test, and you check in the fix, add a note "COMMITTED" with the svn revision (occasionally we'll clean these up)
if you don't have checkin permissions, use the http://bugs.python.org patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)

How to run tests

Assume you've got the py3k branch checked out.

./configure
make
./python Lib/test/regrtest.py test_foobar  # runs test_foobar
./python Lib/test/test_foobar.py   # for more detail about failures

(Some tests need you to use regrtest.py -uall above.)

The Failing Tests

test_ossaudiodev - AssertionError: elapsed time > 10% off of expected time (Note: this is the ultimate flakey test; that assert is just bogus. GvR)

Tests Failing Under Stress

A few tests pass normally but fail when run by a debug build using regrtest.py -R4:3:. This runs the test 7 times to detect leaks. The following tests have problems under this mode:

Confirmed failing tests:
- test_pkg - test_6 and test_7 fail on the second run of "./python Lib/test/regrtest.py --verbose -R4:3: test_pkg"
Leaking tests:
- test_urllib2_localnet (leaks random amounts)
- test_ctypes - [0, 33, -33] references, sum=0
- test_datetime - [49, 49, 49] references, sum=147
- test_io - [62, 62, 62] references, sum=186
- test_strptime - [2132, 2132, 2132] references, sum=6396
- test_string - [5, 5, 5] references, sum=15
- test_urllib - [122, 122, 122] references, sum=366
- test_xmlrpc - [26, 26, 26] references, sum=78

Tests Failing on Some Platforms Only

(Can anyone confirm that these still fail?)

test_ossaudiodev (64-bit)
test_coding (cygwin and some gentoo) - UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 105: ordinal not in range(128)
test_marshal (cygwin)
test_normalization (gentoo) - same as test_coding. On the machine they fail on, locale.getlocale() is (None, None)

-  ⇤ ← Revision 177 as of 2007-08-10 19:16:49 → 
  Size: 8876
  Editor: 65
  Comment:
+   ← Revision 222 as of 2007-08-26 20:10:31 → ⇥
  Size: 2581
  Editor: 72
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 5:
-'''NOTE:''' we've switched to a new branch.  The old py3k-struni branch is dead.  If you have that one checked out, ''don't update'', but issue this command (slightly different for anonymous checkouts):
{{{svn switch svn+ssh://pythondev@svn.python.org/python/branches/py3k}}}
+We're now maintaining a list of tasks in a shared spreadsheet: http://spreadsheets.google.com/ccc?key=pBLWM8elhFAmKbrhhh0ApQA
-Line 13:
+Line 12:
-  * if you don't have checkin permissions, use the SF patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)
+  * if you don't have checkin permissions, use the http://bugs.python.org patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)
-Line 19:
+Line 18:
-'''NOTE:''' we've switched to a new branch.  The old py3k-struni branch is dead.  See the note at the top of this page.
-Line 33:
+Line 31:
-test_bsddb3

test_email (CosminStejerean - at least some problems seem to be caused by base64MIME, problems with splitting headers into multiple lines - 08/08/07)

test_email_codecs

test_email_renamed
+  * test_ossaudiodev - AssertionError: elapsed time > 10% off of expected time (Note: this is the ultimate flakey test; that assert is just bogus. GvR)
-Line 46:
+Line 37:
-  * Tests that pass but report leaks:

    * test_array leaked [11, 11, 11] references, sum=33
    * test_multibytecodec leaked [72, 72, 72] references, sum=216
    * test_parser leaked [5, 5, 5] references, sum=15
    * test_zipimport leaked [29, 29, 29] references, sum=87
-Line 55:
+Line 39:
-    * test_structmember
    * test_datetime (segfaults when run together with others)
    * test_collections
    * test_runpy
    * test_asynchat (seems to hang in cleanup)
+    * test_pkg - test_6 and test_7 fail on the second run of "./python Lib/test/regrtest.py --verbose -R4:3: test_pkg"
-Line 61:
+Line 41:
-  * Uncofirmed failures:
+  * Leaking tests:
-Line 63:
+Line 43:
-    * test_shelve?
    * test_gzip?
    * test_poplib? (Neal reported that it pegged the CPU)
+    * test_urllib2_localnet (leaks random amounts)
    * test_ctypes - [0, 33, -33] references, sum=0
    * test_datetime - [49, 49, 49] references, sum=147
    * test_io - [62, 62, 62] references, sum=186
    * test_strptime - [2132, 2132, 2132] references, sum=6396
    * test_string - [5, 5, 5] references, sum=15
    * test_urllib - [122, 122, 122] references, sum=366
    * test_xmlrpc - [26, 26, 26] references, sum=78
-Line 70:
+Line 54:
+(Can anyone confirm that these still fail?)
-Line 71:
+Line 56:
-=== Failing on 64-bit Platforms Only ===
+  * test_ossaudiodev (64-bit)
-Line 73:
+Line 58:
-test_ossaudiodev (64-bit: ???)
+  * test_coding (cygwin and some gentoo) - UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 105: ordinal not in range(128)
-Line 75:
+Line 60:
-test_shelve (64-bit: ?)
+  * test_marshal (cygwin)
-Line 77:
+Line 62:
-=== Failing on CYGWIN Only ===

test_coding

test_marshal

------

= Previously Fixed Tests =

python -v crashes due to infinite recursion; FIXED (w. a hack) GvR.

test_aepack (OSX only) FIXED IN SF (but somewhat hackily): http://python.org/sf/1761465 JeffreyYasskin 2007-07-26

test_applesingle (OSX only) FIXED IN SF: http://python.org/sf/1761465 JeffreyYasskin 2007-07-26

test_audioop: Fixed in r56469. MvL

test_asynchat FIXED IN SF http://www.python.org/sf/1752173 Alexandre Vassalotti. COMMITTED r56290.

test_bigmem FIXED IN SF http://www.python.org/sf/1752195. COMMITTED r56295.

test_bsddb: Fixed in r56754. MvL

test_cmd_line: FIXED IN SF http://www.python.org/sf/1751493 ChristianHeimes -- COMMITTED

test_compile: COMMITTED r56494 GvR.

test_cookielib FIXED IN SF http://www.python.org/sf/1762940 JoeGregorio. COMMITTED r56709 GvR.

test_cpickle: Deleted, with cPickle. Alexandre's SoC work should lead to cPickle becoming invisible (Brett Cannon).

test_cProfile: FIXED IN SF http://www.python.org/sf/1755176 by Amaury Forgeot d'Arc COMMITTED r56406.

test_csv - Adam Hupp <firstname at lastname.org> 2007-07-21. COMMITTED GvR.

test_ctypes: ThomasHeller: FIXED r56338.

test_ctypes (another failure; Victor Stinner posted a patch to python-3000) COMMITTED r56838 Thomas Heller.

test_curses (add -uall)COMMITTED r56516 GvR.

test_datetime: pickling is broken since r56461, due to cPickle not understanding bytes. Decided the right solution is to stop using cPickle. COMMITTED r56462. GvR.

test_descr GuidoVanRossum: FIXED r56252.
test_descr says: ['foo\u1234bar'] slots not caught; FIXED r56470 GvR.

test_dl FIXED IN SF http://www.python.org/sf/1752229 AlexandreVassalotti 2007-07-11 COMMITTED r56288.

test_doctest: Fixed for OSX in r56468, MvL. Fixed for Ubuntu in r56472, Collin Winter.

test_frozen COMMITTED r56503 GvR.

test_ftplib: FIXED IN SF http://www.python.org/sf/1755206 by Amaury Forgeot d'Arc. FIXED differently by GvR.

test_getargs: (amauryfa) I suggest to remove this test. Now that conversion from unicode to char* uses utf8, no error can occur.
Deletion COMMITTED r56495 GvR.

test_gettext FIXED IN SF http://www.python.org/sf/1751958 ChristianHeimes. COMMITTED r56292.

test_largefile: converted some string literals to byte literals. COMMITTED r56499 Collin Winter.

test_macostools (OSX only) FIXED IN SF http://python.org/sf/1758570 by Jeffrey Yasskin 22 July 2007 COMMITTED r56510 GvR.

test_mailbox FIXED IN SF: http://python.org/sf/1757774 Alexandre Vassalotti 20 July 2007. COMMITTED r56474 GvR.

test_mmap JoeGregorio FIXED IN SF http://www.python.org/sf/1752647
COMMITTED r56400.

test_minidom PARTLY FIXED IN SF http://python.org/sf/1759922. COMMITTED r56586 GvR. Remaining issue COMMITTED r56587 GvR.

test_old_mailbox FIXED IN SF: http://python.org/sf/1757774 Alexandre Vassalotti 20 July 2007. COMMITTED r56474 GvR.

test_pep263 ChristianHeimes FIXED IN WIKI. The fix isn't worth a patch. You have to convert two string to byte by putting a b in front of them. .encode() returns bytes.  COMMITTED by ThomasHeller.

test_pickle: started complaining about unhashable type: 'bytes' since MvL's checkin. COMMITTED r56461 GvR.

test_pickletools Pickle doesn't handle str vs. str8 vs. byte correctly. Do we pickle str as UNICODE or STRING?  FIXED by GvR.

test_plistlib: [http://python.org/sf/1769016 FIXED IN SF (1769016)] James Brotchie 2007-08-07. COMMITTED by Neal Norwitz.

test_poll Christian Heimes FIXED IN WIKI: Replace MSG = " This is a test." with MSG = b" This is a test."  COMMITTED r56285.

test_poplib: needed to change one str literal to a bytes literal in poplib.py. COMMITTED r56498 Collin Winter.

test_pyexpat  JoeGregorio FIXED IN SF http://www.python.org/sf/1759016. COMMITTED r56512 GvR.

test_quopri: fixed in r56606.

test_resource: fixed by a change to io.py (ignore errors from flush() in close()) COMMITTED r56496 GvR.

test_scriptpackages: [http://python.org/sf/1768976 FIXED IN SF (1768976)] Antti Rasinen 2007-08-07. COMMITTEG r56800 GvR.

test_sgmllib (passes on Ubuntu, fails on OSX, and on CYGWIN) COMMITTED  r56585 GvR.

test_shelve: Requires bsddb to be fixed (Brett Cannon); some work done in r56548.  (Abort trap in dbmmodile.c:107.)  FIX COMMITTED GvR.

test_socketserver (add -uall) FIXED IN SF http://python.org/sf/1764815 Paul Colomiets. COMMITTED r56708 GvR.

test_socket_ssl (only fails when called via regrtest) (FIXED somehow)

test_sqlite Fixed in revision 56902. GerhardHaering. (And 56903, GvR)

test_tarfile: Lars Gustäbel COMMITTED.

test_tcl: COMMITTED r56514 GvR.

test_threaded_import FIXED IN WIKI replace "_os.fdopen" with "_io.open" in Lib/tempfile.py Paul Colomiets <pc at gafol.net>; COMMITTED r56757.

test_ucn FIXED IN SF: http://python.org/sf/1757758 Alexandre Vassalotti 20 July 2007. COMMITTED r56473 GvR.

test_unicode_file: COMMITTED r56625 (Brett Cannon).

test_urllib: COMMITTED r56734 (last of several) (Jeremy Hylton)

test_urllib2: FIXED IN SF http://www.python.org/sf/1762940 JoeGregorio. COMMITTED r56709 gvR.

test_urllib2net (add -uall): MINE Jeremy Hylton 8/04/07 COMMITTED r56837.

test_urllib2_localnet: ISSUE bytes/str issues in BaseHTTPServer and httplib.  MINE Jeremy Hylton 08/03/07 FIXED r56905.

test_urllibnet (add -uall): FIX COMMITTED (Jeremy)

test_uu FIXED IN SF ChristianHeimes http://www.python.org/sf/1754339 The patch may fix several other unit tests, too. (A different fix was committed; GvR.)

test_uuid: COMMITTED r56471. GvR.

test_xdrlib: COMMITTED r56549.

test_xml_etree  FIXED IN SF http://www.python.org/sf/1762412 JoeGregorio COMMITTED

test_xml_etree_c  FIXED IN SF http://python.org/sf/1769767 JoeGregorio. COMMITTED by Neal, r56824.

test_xmlrpc: COMMITTED r56609, 56610, and 56626 (Brett Cannon)

test_xpickle: deleted, with cPickle.

test_wave: probably not worth fixing; aren't we going to delete this?  Anthony Baxter might want it (Brett Cannon). A fix that makes the test passed was COMMITTED r56515 GvR.

test_wsgiref COMMITTED r56836 GvR.

test_zipimport: FIXED IN SF http://www.python.org/sf/1766592 Paul Colomiets. COMMITTED r56707 GvR.
+  * test_normalization (gentoo) - same as test_coding.  On the machine they fail on, locale.getlocale() is (None, None)

Page

User

List of tests that are failing in the py3k branch

How to run tests

The Failing Tests

Tests Failing Under Stress

Tests Failing on Some Platforms Only