List of tests that are failing in the py3k branch

We're now maintaining a list of tasks in a shared spreadsheet: http://spreadsheets.google.com/ccc?key=pBLWM8elhFAmKbrhhh0ApQA

Protocol:

if you start working on a test, to avoid duplicate work, add your name+email+date to the bulleted item for the test below, with the text "MINE"
if you figure out why a test is failing, just add a note (name etc. is still appreciated)
if you fix a test, and you check in the fix, add a note "COMMITTED" with the svn revision (occasionally we'll clean these up)
if you don't have checkin permissions, use the http://bugs.python.org patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)

How to run tests

Assume you've got the py3k branch checked out.

./configure
make
./python Lib/test/regrtest.py test_foobar  # runs test_foobar
./python Lib/test/test_foobar.py   # for more detail about failures

(Some tests need you to use regrtest.py -uall above.)

The Failing Tests

test_ossaudiodev - AssertionError: elapsed time > 10% off of expected time (Note: this is the ultimate flakey test; that assert is just bogus. GvR)

Tests Failing Under Stress

A few tests pass normally but fail when run by a debug build using regrtest.py -R4:3:. This runs the test 7 times to detect leaks. The following tests have problems under this mode:

Confirmed failing tests:
- None
Unconfirmed failing tests:
- test_tcl - _tkinter.TclError: Calling Tk_Init again after a previous call failed might deadlock
- test_pkg - test_6 and test_7 fail on the second run of "./python Lib/test/regrtest.py --verbose -R4:3: test_pkg"
Leaking tests:
- test_urllib2_localnet (leaks random amounts)
- test_ctypes - [0, 33, -33] references, sum=0
- test_datetime - [49, 49, 49] references, sum=147
- test_io - [62, 62, 62] references, sum=186
- test_strptime - [2132, 2132, 2132] references, sum=6396
- test_string - [5, 5, 5] references, sum=15
- test_urllib - [122, 122, 122] references, sum=366
- test_xmlrpc - [26, 26, 26] references, sum=78

Tests Failing on Some Platforms Only

(Can anyone confirm that these still fail?)

test_ossaudiodev (64-bit)
test_coding (cygwin and some gentoo) - UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 105: ordinal not in range(128)
test_marshal (cygwin)
test_normalization (gentoo) - same as test_coding. On the machine they fail on, locale.getlocale() is (None, None)

-  ⇤ ← Revision 162 as of 2007-08-08 16:41:31 → 
  Size: 7517
  Editor: 65
  Comment:
+   ← Revision 220 as of 2007-08-26 19:41:54 → ⇥
  Size: 2732
  Editor: 72
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 3:
-= List of tests that are failing in the py3k-struni branch =
+= List of tests that are failing in the py3k branch =

We're now maintaining a list of tasks in a shared spreadsheet: http://spreadsheets.google.com/ccc?key=pBLWM8elhFAmKbrhhh0ApQA
-Line 10:
+Line 12:
-  * if you don't have checkin permissions, use the SF patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)
+  * if you don't have checkin permissions, use the http://bugs.python.org patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)
-Line 14:
+Line 16:
-Assume you've got the py3k-struni branch checked out.
+Assume you've got the py3k branch checked out.
-Line 28:
+Line 31:
-python -v crashes due to infinite recursion (well, it tried to be infinite until it got a stack overflow :-)
+  * test_ossaudiodev - AssertionError: elapsed time > 10% off of expected time (Note: this is the ultimate flakey test; that assert is just bogus. GvR)
-Line 30:
+Line 33:
-test_bsddb3
+== Tests Failing Under Stress ==
-Line 32:
+Line 35:
-test_ctypes (one test started failing again recently)
+A few tests pass normally but fail when run by a debug build using {{{regrtest.py -R4:3:}}}.  This runs the test 7 times to detect leaks.  The following tests have problems under this mode:
-Line 34:
+Line 37:
-test_email
+  * Confirmed failing tests:
-Line 36:
+Line 39:
-test_email_codecs
+    * None
-Line 38:
+Line 41:
-test_email_renamed
+  * Unconfirmed failing tests:
-Line 40:
+Line 43:
-test_sqlite (MINE GerhardHaering - I will take care of this the next days.)
+    * test_tcl - _tkinter.TclError: Calling Tk_Init again after a previous call failed might deadlock
    * test_pkg - test_6 and test_7 fail on the second run of "./python Lib/test/regrtest.py --verbose -R4:3: test_pkg"
-Line 42:
+Line 46:
-test_wsgiref (used to be skipped, now fails) MINE GvR
+  * Leaking tests:
-Line 44:
+Line 48:
-test_urllib2_localnet: ISSUE bytes/str issues in BaseHTTPServer and httplib.  MINE Jeremy Hylton 08/03/07  (might cover the urllib2?net tests, too)

test_urllib2net (add -uall): MINE Jeremy Hylton 8/04/07
+    * test_urllib2_localnet (leaks random amounts)
    * test_ctypes - [0, 33, -33] references, sum=0
    * test_datetime - [49, 49, 49] references, sum=147
    * test_io - [62, 62, 62] references, sum=186
    * test_strptime - [2132, 2132, 2132] references, sum=6396
    * test_string - [5, 5, 5] references, sum=15
    * test_urllib - [122, 122, 122] references, sum=366
    * test_xmlrpc - [26, 26, 26] references, sum=78
-Line 51:
+Line 59:
+(Can anyone confirm that these still fail?)
-Line 52:
+Line 61:
-=== Failing on 64-bit Platforms Only ===
+  * test_ossaudiodev (64-bit)
-Line 54:
+Line 63:
-test_ossaudiodev (64-bit: ???)
+  * test_coding (cygwin and some gentoo) - UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 105: ordinal not in range(128)
-Line 56:
+Line 65:
-test_shelve (64-bit: ?)
+  * test_marshal (cygwin)
-Line 58:
+Line 67:
-=== Failing on CYGWIN Only ===

test_coding

test_marshal

------

= Previously Fixed Tests =

test_aepack (OSX only) FIXED IN SF (but somewhat hackily): http://python.org/sf/1761465 JeffreyYasskin 2007-07-26

test_applesingle (OSX only) FIXED IN SF: http://python.org/sf/1761465 JeffreyYasskin 2007-07-26

test_audioop: Fixed in r56469. MvL

test_asynchat FIXED IN SF http://www.python.org/sf/1752173 Alexandre Vassalotti. COMMITTED r56290.

test_bigmem FIXED IN SF http://www.python.org/sf/1752195. COMMITTED r56295.

test_bsddb: Fixed in r56754. MvL

test_cmd_line: FIXED IN SF http://www.python.org/sf/1751493 ChristianHeimes -- COMMITTED

test_compile: COMMITTED r56494 GvR.

test_cookielib FIXED IN SF http://www.python.org/sf/1762940 JoeGregorio. COMMITTED r56709 GvR.

test_cpickle: Deleted, with cPickle. Alexandre's SoC work should lead to cPickle becoming invisible (Brett Cannon).

test_cProfile: FIXED IN SF http://www.python.org/sf/1755176 by Amaury Forgeot d'Arc COMMITTED r56406.

test_csv - Adam Hupp <firstname at lastname.org> 2007-07-21. COMMITTED GvR.

test_ctypes: ThomasHeller: FIXED r56338.

test_curses (add -uall)COMMITTED r56516 GvR.

test_datetime: pickling is broken since r56461, due to cPickle not understanding bytes. Decided the right solution is to stop using cPickle. COMMITTED r56462. GvR.

test_descr GuidoVanRossum: FIXED r56252.
test_descr says: ['foo\u1234bar'] slots not caught; FIXED r56470 GvR.

test_dl FIXED IN SF http://www.python.org/sf/1752229 AlexandreVassalotti 2007-07-11 COMMITTED r56288.

test_doctest: Fixed for OSX in r56468, MvL. Fixed for Ubuntu in r56472, Collin Winter.

test_frozen COMMITTED r56503 GvR.

test_ftplib: FIXED IN SF http://www.python.org/sf/1755206 by Amaury Forgeot d'Arc. FIXED differently by GvR.

test_getargs: (amauryfa) I suggest to remove this test. Now that conversion from unicode to char* uses utf8, no error can occur.
Deletion COMMITTED r56495 GvR.

test_gettext FIXED IN SF http://www.python.org/sf/1751958 ChristianHeimes. COMMITTED r56292.

test_largefile: converted some string literals to byte literals. COMMITTED r56499 Collin Winter.

test_macostools (OSX only) FIXED IN SF http://python.org/sf/1758570 by Jeffrey Yasskin 22 July 2007 COMMITTED r56510 GvR.

test_mailbox FIXED IN SF: http://python.org/sf/1757774 Alexandre Vassalotti 20 July 2007. COMMITTED r56474 GvR.

test_mmap JoeGregorio FIXED IN SF http://www.python.org/sf/1752647
COMMITTED r56400.

test_minidom PARTLY FIXED IN SF http://python.org/sf/1759922. COMMITTED r56586 GvR. Remaining issue COMMITTED r56587 GvR.

test_old_mailbox FIXED IN SF: http://python.org/sf/1757774 Alexandre Vassalotti 20 July 2007. COMMITTED r56474 GvR.

test_pep263 ChristianHeimes FIXED IN WIKI. The fix isn't worth a patch. You have to convert two string to byte by putting a b in front of them. .encode() returns bytes.  COMMITTED by ThomasHeller.

test_pickle: started complaining about unhashable type: 'bytes' since MvL's checkin. COMMITTED r56461 GvR.

test_pickletools Pickle doesn't handle str vs. str8 vs. byte correctly. Do we pickle str as UNICODE or STRING?  FIXED by GvR.

test_plistlib: [http://python.org/sf/1769016 FIXED IN SF (1769016)] James Brotchie 2007-08-07. COMMITTED by Neal Norwitz.

test_poll Christian Heimes FIXED IN WIKI: Replace MSG = " This is a test." with MSG = b" This is a test."  COMMITTED r56285.

test_poplib: needed to change one str literal to a bytes literal in poplib.py. COMMITTED r56498 Collin Winter.

test_pyexpat  JoeGregorio FIXED IN SF http://www.python.org/sf/1759016. COMMITTED r56512 GvR.

test_quopri: fixed in r56606.

test_resource: fixed by a change to io.py (ignore errors from flush() in close()) COMMITTED r56496 GvR.

test_scriptpackages: [http://python.org/sf/1768976 FIXED IN SF (1768976)] Antti Rasinen 2007-08-07. COMMITTEG r56800 GvR.

test_sgmllib (passes on Ubuntu, fails on OSX, and on CYGWIN) COMMITTED  r56585 GvR.

test_shelve: Requires bsddb to be fixed (Brett Cannon); some work done in r56548.  (Abort trap in dbmmodile.c:107.)  FIX COMMITTED GvR.

test_socketserver (add -uall) FIXED IN SF http://python.org/sf/1764815 Paul Colomiets. COMMITTED r56708 GvR.

test_socket_ssl (only fails when called via regrtest) (FIXED somehow)

test_tarfile: Lars Gustäbel COMMITTED.

test_tcl: COMMITTED r56514 GvR.

test_threaded_import FIXED IN WIKI replace "_os.fdopen" with "_io.open" in Lib/tempfile.py Paul Colomiets <pc at gafol.net>; COMMITTED r56757.

test_ucn FIXED IN SF: http://python.org/sf/1757758 Alexandre Vassalotti 20 July 2007. COMMITTED r56473 GvR.

test_unicode_file: COMMITTED r56625 (Brett Cannon).

test_urllib: COMMITTED r56734 (last of several) (Jeremy Hylton)

test_urllib2: FIXED IN SF http://www.python.org/sf/1762940 JoeGregorio. COMMITTED r56709 gvR.

test_urllibnet (add -uall): FIX COMMITTED (Jeremy)

test_uu FIXED IN SF ChristianHeimes http://www.python.org/sf/1754339 The patch may fix several other unit tests, too. (A different fix was committed; GvR.)

test_uuid: COMMITTED r56471. GvR.

test_xdrlib: COMMITTED r56549.

test_xml_etree  FIXED IN SF http://www.python.org/sf/1762412 JoeGregorio COMMITTED

test_xml_etree_c  FIXED IN SF http://python.org/sf/1769767 JoeGregorio. COMMITTED by Neal, r56824.

test_xmlrpc: COMMITTED r56609, 56610, and 56626 (Brett Cannon)

test_xpickle: deleted, with cPickle.

test_wave: probably not worth fixing; aren't we going to delete this?  Anthony Baxter might want it (Brett Cannon). A fix that makes the test passed was COMMITTED r56515 GvR.

test_zipimport: FIXED IN SF http://www.python.org/sf/1766592 Paul Colomiets. COMMITTED r56707 GvR.
+  * test_normalization (gentoo) - same as test_coding.  On the machine they fail on, locale.getlocale() is (None, None)

Page

User

List of tests that are failing in the py3k branch

How to run tests

The Failing Tests

Tests Failing Under Stress

Tests Failing on Some Platforms Only