List of tests that are failing in the py3k branch

We're now maintaining a list of tasks in a shared spreadsheet: http://spreadsheets.google.com/ccc?key=pBLWM8elhFAmKbrhhh0ApQA

Protocol:

if you start working on a test, to avoid duplicate work, add your name+email+date to the bulleted item for the test below, with the text "MINE"
if you figure out why a test is failing, just add a note (name etc. is still appreciated)
if you fix a test, and you check in the fix, add a note "COMMITTED" with the svn revision (occasionally we'll clean these up)
if you don't have checkin permissions, use the http://bugs.python.org patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)

How to run tests

Assume you've got the py3k branch checked out.

./configure
make
./python Lib/test/regrtest.py test_foobar  # runs test_foobar
./python Lib/test/test_foobar.py   # for more detail about failures

(Some tests need you to use regrtest.py -uall above.)

The Failing Tests

test_ossaudiodev - AssertionError: elapsed time > 10% off of expected time (Note: this is the ultimate flakey test; that assert is just bogus. GvR)

Tests Failing Under Stress

A few tests pass normally but fail when run by a debug build using regrtest.py -R4:3:. This runs the test 7 times to detect leaks. The following tests have problems under this mode:

Confirmed failing tests:
- test_pkg - test_6 and test_7 fail on the second run of "./python Lib/test/regrtest.py --verbose -R4:3: test_pkg"
Leaking tests:
- test_urllib2_localnet (leaks random amounts)
- test_ctypes - [0, 33, -33] references, sum=0
- test_datetime - [49, 49, 49] references, sum=147
- test_io - [62, 62, 62] references, sum=186
- test_strptime - [2132, 2132, 2132] references, sum=6396
- test_string - [5, 5, 5] references, sum=15
- test_urllib - [122, 122, 122] references, sum=366
- test_xmlrpc - [26, 26, 26] references, sum=78

Tests Failing on Some Platforms Only

(Can anyone confirm that these still fail?)

test_ossaudiodev (64-bit)
test_coding (cygwin and some gentoo) - UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 105: ordinal not in range(128)
test_marshal (cygwin)
test_normalization (gentoo) - same as test_coding. On the machine they fail on, locale.getlocale() is (None, None)

-  ⇤ ← Revision 66 as of 2007-07-20 17:02:54 → 
  Size: 4640
  Editor: 65
  Comment:
+   ← Revision 222 as of 2007-08-26 20:10:31 → ⇥
  Size: 2581
  Editor: 72
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-= List of tests that are failing in the py3k-struni branch =
+(Back to ["Python3000"].)
 Line 3:
-(Back to ["Python3000"].)
+= List of tests that are failing in the py3k branch =

We're now maintaining a list of tasks in a shared spreadsheet: http://spreadsheets.google.com/ccc?key=pBLWM8elhFAmKbrhhh0ApQA
-Line 10:
+Line 12:
-  * if you don't have checkin permissions, use the SF patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)
+  * if you don't have checkin permissions, use the http://bugs.python.org patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)
-Line 14:
+Line 16:
-Assume you've got the py3k-struni branch checked out.
+Assume you've got the py3k branch checked out.
-Line 22:
+Line 25:
-== The tests ==
+(Some tests need you to use {{{regrtest.py -uall}}} above.)
-Line 24:
+Line 27:
-test_aepack (OSX only)
+------
-Line 26:
+Line 29:
-test_applesingle (OSX only)
+= The Failing Tests =
-Line 28:
+Line 31:
-test_bsddb
+  * test_ossaudiodev - AssertionError: elapsed time > 10% off of expected time (Note: this is the ultimate flakey test; that assert is just bogus. GvR)
-Line 30:
+Line 33:
-test_compile
+== Tests Failing Under Stress ==
-Line 32:
+Line 35:
-test_coding: fails on CYGWIN only
+A few tests pass normally but fail when run by a debug build using {{{regrtest.py -R4:3:}}}.  This runs the test 7 times to detect leaks.  The following tests have problems under this mode:
-Line 34:
+Line 37:
-test_cookielib
+  * Confirmed failing tests:
-Line 36:
+Line 39:
-test_csv
+    * test_pkg - test_6 and test_7 fail on the second run of "./python Lib/test/regrtest.py --verbose -R4:3: test_pkg"
-Line 38:
+Line 41:
-test_descr says: ['foo\u1234bar'] slots not caught
+  * Leaking tests:
-Line 40:
+Line 43:
-test_doctest: Fixed in r56468. MvL; still broken on ubuntu GvR.
+    * test_urllib2_localnet (leaks random amounts)
    * test_ctypes - [0, 33, -33] references, sum=0
    * test_datetime - [49, 49, 49] references, sum=147
    * test_io - [62, 62, 62] references, sum=186
    * test_strptime - [2132, 2132, 2132] references, sum=6396
    * test_string - [5, 5, 5] references, sum=15
    * test_urllib - [122, 122, 122] references, sum=366
    * test_xmlrpc - [26, 26, 26] references, sum=78
-Line 42:
+Line 52:
-test_email
+== Tests Failing on Some Platforms Only ==
-Line 44:
+Line 54:
-test_email_codecs
+(Can anyone confirm that these still fail?)
-Line 46:
+Line 56:
-test_email_renamed
+  * test_ossaudiodev (64-bit)
-Line 48:
+Line 58:
-test_frozen in Python/frozen.c ''{"__phello__.spam", M___hello__, SIZE},'' doesn't add a spam module.  (But why?!?!)
+  * test_coding (cygwin and some gentoo) - UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 105: ordinal not in range(128)
-Line 50:
+Line 60:
-test_getargs: (amauryfa) I suggest to remove this test. Now that conversion from unicode to char* uses utf8, no error can occur.
+  * test_marshal (cygwin)
-Line 52:
+Line 62:
-test_largefile

test_macostools (OSX only)

test_mailbox

test_marshal: fails on CYGWIN only

test_minidom

test_old_mailbox

test_plistlib (OSX only)

test_poplib

test_pyexpat

test_quopri

test_resource

test_scriptpackages (OSX only)

test_sgmllib (passes on Ubuntu, fails on OSX, and on CYGWIN)

test_shelve

test_sqlite

test_tarfile: A lot of TypeError: can't concat bytes to str and other  byte/str confusions. 

test_tcl

test_threaded_import

test_ucn

test_unicode_file

test_urllib2: tests pass on my laptop, don't pass elsewhere -- giving up as I don't know what's up with it -- Hasan Diwan.

test_urllib2_localnet

test_uuid (passes on OSX, fails on Ubuntu), Guido has disabled some tests: 53686 guido.van.rossum         print("""    WARNING: uuid._unixdll_getnode is unreliable on many platforms. 

test_wave: probably not worth fixing; aren't we going to delete this?  Anthony Baxter might want it (Brett Cannon).

test_xdrlib

test_xml_etree

test_xml_etree_c

test_xmlrpc

test_zipimport: lots of str/bytes issues, shouldn't be too hard; important

== Previously Fixed ==

test_audioop: Fixed in r56469. MvL

test_asynchat FIXED IN SF http://www.python.org/sf/1752173 Alexandre Vassalotti. COMMITTED r56290.

test_bigmem FIXED IN SF http://www.python.org/sf/1752195. COMMITTED r56295.

test_cmd_line: FIXED IN SF http://www.python.org/sf/1751493 ChristianHeimes -- COMMITTED
 
test_cpickle: Deleted, with cPickle. Alexandre's SoC work should lead to cPickle becoming invisible (Brett Cannon).

test_ctypes: ThomasHeller: FIXED r56338.

test_cProfile: FIXED IN SF http://www.python.org/sf/1755176 by Amaury Forgeot d'Arc COMMITTED r56406.

test_datetime: pickling is broken since r56461, due to cPickle not understanding bytes. Decided the right solution is to stop using cPickle. COMMITTED r56462. GvR.

test_descr GuidoVanRossum: FIXED r56252.

test_dl FIXED IN SF http://www.python.org/sf/1752229 AlexandreVassalotti 2007-07-11 COMMITTED r56288.

test_ftplib: FIXED IN SF http://www.python.org/sf/1755206 by Amaury Forgeot d'Arc. FIXED differently by GvR.

test_gettext FIXED IN SF http://www.python.org/sf/1751958 ChristianHeimes. COMMITTED r56292.

test_mmap JoeGregorio FIXED IN SF http://www.python.org/sf/1752647
COMMITTED r56400.

test_pep263 ChristianHeimes FIXED IN WIKI. The fix isn't worth a patch. You have to convert two string to byte by putting a b in front of them. .encode() returns bytes.  COMMITTED by ThomasHeller.

test_pickle: started complaining about unhashable type: 'bytes' since MvL's checkin. COMMITTED r56461 GvR.

test_pickletools Pickle doesn't handle str vs. str8 vs. byte correctly. Do we pickle str as UNICODE or STRING?  FIXED by GvR.

test_poll Christian Heimes FIXED IN WIKI: Replace MSG = " This is a test." with MSG = b" This is a test."  COMMITTED r56285.

test_uu FIXED IN SF ChristianHeimes http://www.python.org/sf/1754339 The patch may fix several other unit tests, too. (A different fix was committed; GvR.)

test_xpickle: deleted, with cPickle.
+  * test_normalization (gentoo) - same as test_coding.  On the machine they fail on, locale.getlocale() is (None, None)

Page

User

List of tests that are failing in the py3k branch

How to run tests

The Failing Tests

Tests Failing Under Stress

Tests Failing on Some Platforms Only