Differences between revisions 7 and 222 (spanning 215 versions)
Revision 7 as of 2007-07-10 20:59:58
Size: 2192
Editor: 216
Comment: test_file is fixed
Revision 222 as of 2007-08-26 20:10:31
Size: 2581
Editor: 72
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= List of tests that are failing in the py3k-struni branch = (Back to ["Python3000"].)
Line 3: Line 3:
(Back to ["Python3000"].) = List of tests that are failing in the py3k branch =

We're now maintaining a list of tasks in a shared spreadsheet: http://spreadsheets.google.com/ccc?key=pBLWM8elhFAmKbrhhh0ApQA
Line 7: Line 9:
  * if you start working on a test, to avoid duplicate work, add your name+email+date to the bulleted item for the test below, with the text "FIXING" or "INVESTIGATING"   * if you start working on a test, to avoid duplicate work, add your name+email+date to the bulleted item for the test below, with the text "MINE"
Line 9: Line 11:
  * if you fix a test, and you check in the fix, add a note "FIXED" (occasionally we'll clean these up)
  * if you don't have checkin permissions, use the SF patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED" (plus name etc.)
  * if you fix a test, and you check in the fix, add a note "COMMITTED" with the svn revision (occasionally we'll clean these up)
  * if you don't have checkin permissions, use the http://bugs.python.org patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)
Line 14: Line 16:
Assume you've got the py3k-struni branch checked out. Assume you've got the py3k branch checked out.
Line 22: Line 25:
== The tests == (Some tests need you to use {{{regrtest.py -uall}}} above.)
Line 24: Line 27:
test_aepack (OSX only) ------
Line 26: Line 29:
test_applesingle (OSX only) = The Failing Tests =
Line 28: Line 31:
test_asynchat   * test_ossaudiodev - AssertionError: elapsed time > 10% off of expected time (Note: this is the ultimate flakey test; that assert is just bogus. GvR)
Line 30: Line 33:
test_audioop == Tests Failing Under Stress ==
Line 32: Line 35:
test_bigmem A few tests pass normally but fail when run by a debug build using {{{regrtest.py -R4:3:}}}. This runs the test 7 times to detect leaks. The following tests have problems under this mode:
Line 34: Line 37:
test_cProfile: not worth it? (I think this will be killed)   * Confirmed failing tests:
Line 36: Line 39:
test_cmd_line     * test_pkg - test_6 and test_7 fail on the second run of "./python Lib/test/regrtest.py --verbose -R4:3: test_pkg"
Line 38: Line 41:
test_compile   * Leaking tests:
Line 40: Line 43:
test_cookielib     * test_urllib2_localnet (leaks random amounts)
    * test_ctypes - [0, 33, -33] references, sum=0
    * test_datetime - [49, 49, 49] references, sum=147
    * test_io - [62, 62, 62] references, sum=186
    * test_strptime - [2132, 2132, 2132] references, sum=6396
    * test_string - [5, 5, 5] references, sum=15
    * test_urllib - [122, 122, 122] references, sum=366
    * test_xmlrpc - [26, 26, 26] references, sum=78
Line 42: Line 52:
test_cpickle: not worth it? (I think this may be killed.) == Tests Failing on Some Platforms Only ==
Line 44: Line 54:
test_csv (Can anyone confirm that these still fail?)
Line 46: Line 56:
test_ctypes   * test_ossaudiodev (64-bit)
Line 48: Line 58:
test_descr   * test_coding (cygwin and some gentoo) - UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 105: ordinal not in range(128)
Line 50: Line 60:
test_dl   * test_marshal (cygwin)
Line 52: Line 62:
test_doctest

test_email

test_email_codecs

test_email_renamed

test_frozen

test_ftplib

test_getargs

test_gettext

test_macostools (OSX only)

test_mailbox

test_minidom

test_mmap

test_old_mailbox

test_pep263

test_pickletools

test_plistlib (OSX only)

test_poll

test_poplib

test_pyexpat

test_quopri

test_resource

test_scriptpackages (OSX only)

test_sgmllib (passes on Ubuntu, fails on OSX)

test_shelve

test_sqlite

test_tarfile

test_tcl

test_threaded_import

test_ucn

test_unicode_file

test_urllib2

test_urllib2_localnet

test_uu

test_uuid (passes on OSX, fails on Ubuntu)

test_wave: probably not worth fixing; aren't we going to delete this?

test_xdrlib

test_xml_etree

test_xml_etree_c

test_xmlrpc

test_xpickle

test_zipimport: lots of str/bytes issues, shouldn't be too hard; important

----
CategoryLanguage
  * test_normalization (gentoo) - same as test_coding. On the machine they fail on, locale.getlocale() is (None, None)

(Back to ["Python3000"].)

List of tests that are failing in the py3k branch

We're now maintaining a list of tasks in a shared spreadsheet: http://spreadsheets.google.com/ccc?key=pBLWM8elhFAmKbrhhh0ApQA

Protocol:

  • if you start working on a test, to avoid duplicate work, add your name+email+date to the bulleted item for the test below, with the text "MINE"
  • if you figure out why a test is failing, just add a note (name etc. is still appreciated)
  • if you fix a test, and you check in the fix, add a note "COMMITTED" with the svn revision (occasionally we'll clean these up)
  • if you don't have checkin permissions, use the http://bugs.python.org patch manager and note the patch number here (perhaps in the form of a link to python.org/sf/NNNNN), and add a note "FIXED IN SF" (plus name etc.)

How to run tests

Assume you've got the py3k branch checked out.

./configure
make
./python Lib/test/regrtest.py test_foobar  # runs test_foobar
./python Lib/test/test_foobar.py   # for more detail about failures

(Some tests need you to use regrtest.py -uall above.)


The Failing Tests

  • test_ossaudiodev - AssertionError: elapsed time > 10% off of expected time (Note: this is the ultimate flakey test; that assert is just bogus. GvR)

Tests Failing Under Stress

A few tests pass normally but fail when run by a debug build using regrtest.py -R4:3:. This runs the test 7 times to detect leaks. The following tests have problems under this mode:

  • Confirmed failing tests:
    • test_pkg - test_6 and test_7 fail on the second run of "./python Lib/test/regrtest.py --verbose -R4:3: test_pkg"
  • Leaking tests:
    • test_urllib2_localnet (leaks random amounts)
    • test_ctypes - [0, 33, -33] references, sum=0
    • test_datetime - [49, 49, 49] references, sum=147
    • test_io - [62, 62, 62] references, sum=186
    • test_strptime - [2132, 2132, 2132] references, sum=6396
    • test_string - [5, 5, 5] references, sum=15
    • test_urllib - [122, 122, 122] references, sum=366
    • test_xmlrpc - [26, 26, 26] references, sum=78

Tests Failing on Some Platforms Only

(Can anyone confirm that these still fail?)

  • test_ossaudiodev (64-bit)
  • test_coding (cygwin and some gentoo) - UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 105: ordinal not in range(128)

  • test_marshal (cygwin)
  • test_normalization (gentoo) - same as test_coding. On the machine they fail on, locale.getlocale() is (None, None)

Py3kStrUniTests (last edited 2008-11-15 14:00:38 by localhost)

Unable to edit the page? See the FrontPage for instructions.