List of tests that are failing in the py3k branch

We're now maintaining a list of tasks in a shared spreadsheet: http://spreadsheets.google.com/ccc?key=pBLWM8elhFAmKbrhhh0ApQA

Protocol:

if you start working on a test, to avoid duplicate work, add your name+email+date to the bulleted item for the test below, with the text "MINE"
if you figure out why a test is failing, just add a note (name etc. is still appreciated)
if you fix a test, and you check in the fix, add a note "COMMITTED" with the svn revision (occasionally we'll clean these up)
if you don't have checkin permissions, use the http://bugs.python.org patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)

How to run tests

Assume you've got the py3k branch checked out.

./configure
make
./python Lib/test/regrtest.py test_foobar  # runs test_foobar
./python Lib/test/test_foobar.py   # for more detail about failures

(Some tests need you to use regrtest.py -uall above.)

The Failing Tests

test_ossaudiodev - AssertionError: elapsed time > 10% off of expected time (Note: this is the ultimate flakey test; that assert is just bogus. GvR)

Tests Failing Under Stress

A few tests pass normally but fail when run by a debug build using regrtest.py -R4:3:. This runs the test 7 times to detect leaks. The following tests have problems under this mode:

Confirmed failing tests:
- test_pkg - test_6 and test_7 fail on the second run of "./python Lib/test/regrtest.py --verbose -R4:3: test_pkg"
Leaking tests:
- test_urllib2_localnet (leaks random amounts)
- test_ctypes - [0, 33, -33] references, sum=0
- test_datetime - [49, 49, 49] references, sum=147
- test_io - [62, 62, 62] references, sum=186
- test_strptime - [2132, 2132, 2132] references, sum=6396
- test_string - [5, 5, 5] references, sum=15
- test_urllib - [122, 122, 122] references, sum=366
- test_xmlrpc - [26, 26, 26] references, sum=78

Tests Failing on Some Platforms Only

(Can anyone confirm that these still fail?)

test_ossaudiodev (64-bit)
test_coding (cygwin and some gentoo) - UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 105: ordinal not in range(128)
test_marshal (cygwin)
test_normalization (gentoo) - same as test_coding. On the machine they fail on, locale.getlocale() is (None, None)

-  ⇤ ← Revision 118 as of 2007-07-30 01:35:21 → 
  Size: 6429
  Editor: BrettCannon
  Comment: Report revision with test_unicode_file fix.
+   ← Revision 222 as of 2007-08-26 20:10:31 → ⇥
  Size: 2581
  Editor: 72
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-= List of tests that are failing in the py3k-struni branch =
+(Back to ["Python3000"].)
 Line 3:
-(Back to ["Python3000"].)
+= List of tests that are failing in the py3k branch =

We're now maintaining a list of tasks in a shared spreadsheet: http://spreadsheets.google.com/ccc?key=pBLWM8elhFAmKbrhhh0ApQA
-Line 10:
+Line 12:
-  * if you don't have checkin permissions, use the SF patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)
+  * if you don't have checkin permissions, use the http://bugs.python.org patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)
-Line 14:
+Line 16:
-Assume you've got the py3k-struni branch checked out.
+Assume you've got the py3k branch checked out.
-Line 24:
+Line 27:
-== The Failing Tests ==
+------
-Line 26:
+Line 29:
-test_bsddb
+= The Failing Tests =
-Line 28:
+Line 31:
-test_bsddb3
+  * test_ossaudiodev - AssertionError: elapsed time > 10% off of expected time (Note: this is the ultimate flakey test; that assert is just bogus. GvR)
-Line 30:
+Line 33:
-test_coding: fails on CYGWIN only
+== Tests Failing Under Stress ==
-Line 32:
+Line 35:
-test_cookielib FIXED IN SF http://www.python.org/sf/1762940 JoeGregorio
+A few tests pass normally but fail when run by a debug build using {{{regrtest.py -R4:3:}}}.  This runs the test 7 times to detect leaks.  The following tests have problems under this mode:
-Line 34:
+Line 37:
-test_csv - Adam Hupp <firstname at lastname.org> 2007-07-21
+  * Confirmed failing tests:
-Line 36:
+Line 39:
-test_email
+    * test_pkg - test_6 and test_7 fail on the second run of "./python Lib/test/regrtest.py --verbose -R4:3: test_pkg"
-Line 38:
+Line 41:
-test_email_codecs
+  * Leaking tests:
-Line 40:
+Line 43:
-test_email_renamed
+    * test_urllib2_localnet (leaks random amounts)
    * test_ctypes - [0, 33, -33] references, sum=0
    * test_datetime - [49, 49, 49] references, sum=147
    * test_io - [62, 62, 62] references, sum=186
    * test_strptime - [2132, 2132, 2132] references, sum=6396
    * test_string - [5, 5, 5] references, sum=15
    * test_urllib - [122, 122, 122] references, sum=366
    * test_xmlrpc - [26, 26, 26] references, sum=78
-Line 42:
+Line 52:
-test_marshal: fails on CYGWIN only
+== Tests Failing on Some Platforms Only ==
-Line 44:
+Line 54:
-test_plistlib (OSX only)
+(Can anyone confirm that these still fail?)
-Line 46:
+Line 56:
-test_scriptpackages (OSX only)
+  * test_ossaudiodev (64-bit)
-Line 48:
+Line 58:
-test_shelve: WHY = Requires bsddb to be fixed (Brett Cannon); some work done in r56548.
+  * test_coding (cygwin and some gentoo) - UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 105: ordinal not in range(128)
-Line 50:
+Line 60:
-test_socketserver (add -uall)
+  * test_marshal (cygwin)
-Line 52:
+Line 62:
-test_sqlite

test_tarfile: A lot of TypeError: can't concat bytes to str and other  byte/str confusions. 

test_threaded_import FIXED IN WIKI replace "_os.fdopen" with "_io.open" in Lib/tempfile.py Paul Colomiets <pc at gafol.net>

test_unicode_file: FIXED in r56625 (Brett Cannon).

test_urllib2: FIXED IN SF http://www.python.org/sf/1762940 JoeGregorio

test_urllib2_localnet

test_urllib2net (add -uall)

test_urllibnet (add -uall)

test_xml_etree_c

test_xmlrpc: Some work in r56609, 56610. Code works in string literals but calls str.encode() which returns bytes and causes the str.join errors.  Make xmlwprclib.dumps return bytes, or don't perform encoding? (Brett Cannon)

test_zipimport: lots of str/bytes issues, shouldn't be too hard; important

== Previously Fixed Tests ==

test_aepack (OSX only) FIXED IN SF (but somewhat hackily): http://python.org/sf/1761465 JeffreyYasskin 2007-07-26

test_applesingle (OSX only) FIXED IN SF: http://python.org/sf/1761465 JeffreyYasskin 2007-07-26

test_audioop: Fixed in r56469. MvL

test_asynchat FIXED IN SF http://www.python.org/sf/1752173 Alexandre Vassalotti. COMMITTED r56290.

test_bigmem FIXED IN SF http://www.python.org/sf/1752195. COMMITTED r56295.

test_cmd_line: FIXED IN SF http://www.python.org/sf/1751493 ChristianHeimes -- COMMITTED

test_compile: COMMITTED r56494 GvR.

test_cpickle: Deleted, with cPickle. Alexandre's SoC work should lead to cPickle becoming invisible (Brett Cannon).

test_cProfile: FIXED IN SF http://www.python.org/sf/1755176 by Amaury Forgeot d'Arc COMMITTED r56406.

test_ctypes: ThomasHeller: FIXED r56338.

test_curses (add -uall)COMMITTED r56516 GvR.

test_datetime: pickling is broken since r56461, due to cPickle not understanding bytes. Decided the right solution is to stop using cPickle. COMMITTED r56462. GvR.

test_descr GuidoVanRossum: FIXED r56252.
test_descr says: ['foo\u1234bar'] slots not caught; FIXED r56470 GvR.

test_dl FIXED IN SF http://www.python.org/sf/1752229 AlexandreVassalotti 2007-07-11 COMMITTED r56288.

test_doctest: Fixed for OSX in r56468, MvL. Fixed for Ubuntu in r56472, Collin Winter.

test_frozen COMMITTED r56503 GvR.

test_ftplib: FIXED IN SF http://www.python.org/sf/1755206 by Amaury Forgeot d'Arc. FIXED differently by GvR.

test_getargs: (amauryfa) I suggest to remove this test. Now that conversion from unicode to char* uses utf8, no error can occur.
Deletion COMMITTED r56495 GvR.

test_gettext FIXED IN SF http://www.python.org/sf/1751958 ChristianHeimes. COMMITTED r56292.

test_largefile: converted some string literals to byte literals. COMMITTED r56499 Collin Winter.

test_macostools (OSX only) FIXED IN SF http://python.org/sf/1758570 by Jeffrey Yasskin 22 July 2007 COMMITTED r56510 GvR.

test_mailbox FIXED IN SF: http://python.org/sf/1757774 Alexandre Vassalotti 20 July 2007. COMMITTED r56474 GvR.

test_mmap JoeGregorio FIXED IN SF http://www.python.org/sf/1752647
COMMITTED r56400.

test_minidom PARTLY FIXED IN SF http://python.org/sf/1759922. COMMITTED r56586 GvR. Remaining issue COMMITTED r56587 GvR.

test_old_mailbox FIXED IN SF: http://python.org/sf/1757774 Alexandre Vassalotti 20 July 2007. COMMITTED r56474 GvR.

test_pep263 ChristianHeimes FIXED IN WIKI. The fix isn't worth a patch. You have to convert two string to byte by putting a b in front of them. .encode() returns bytes.  COMMITTED by ThomasHeller.

test_pickle: started complaining about unhashable type: 'bytes' since MvL's checkin. COMMITTED r56461 GvR.

test_pickletools Pickle doesn't handle str vs. str8 vs. byte correctly. Do we pickle str as UNICODE or STRING?  FIXED by GvR.

test_poll Christian Heimes FIXED IN WIKI: Replace MSG = " This is a test." with MSG = b" This is a test."  COMMITTED r56285.

test_poplib: needed to change one str literal to a bytes literal in poplib.py. COMMITTED r56498 Collin Winter.

test_pyexpat  JoeGregorio FIXED IN SF http://www.python.org/sf/1759016. COMMITTED r56512 GvR.

test_quopri: fixed in r56606.

test_resource: fixed by a change to io.py (ignore errors from flush() in close()) COMMITTED r56496 GvR.

test_sgmllib (passes on Ubuntu, fails on OSX, and on CYGWIN) COMMITTED  r56585 GvR.

test_tcl: COMMITTED r56514 GvR.

test_ucn FIXED IN SF: http://python.org/sf/1757758 Alexandre Vassalotti 20 July 2007. COMMITTED r56473 GvR.

test_uu FIXED IN SF ChristianHeimes http://www.python.org/sf/1754339 The patch may fix several other unit tests, too. (A different fix was committed; GvR.)

test_uuid: COMMITTED r56471. GvR.

test_xdrlib: COMMITTED r56549.

test_xml_etree  FIXED IN SF http://www.python.org/sf/1762412 JoeGregorio

test_xpickle: deleted, with cPickle.

test_wave: probably not worth fixing; aren't we going to delete this?  Anthony Baxter might want it (Brett Cannon). A fix that makes the test passed was COMMITTED r56515 GvR.
+  * test_normalization (gentoo) - same as test_coding.  On the machine they fail on, locale.getlocale() is (None, None)

Page

User

List of tests that are failing in the py3k branch

How to run tests

The Failing Tests

Tests Failing Under Stress

Tests Failing on Some Platforms Only