Differences between revisions 44 and 45
⇤ ← Revision 44 as of 2008-01-04 04:29:51 →
Size: 9664
Editor: c-67-177-212-165
Comment: Added what we need to get modern into trunk
← Revision 45 as of 2008-01-04 04:58:20 → ⇥
Size: 10538
Editor: c-67-177-212-165
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
== Gaps between Jython and Django == [http://pipes.yahoo.com/pipes/pipe.info?_id=jroxOD1f3BGgvtiIqGIyXQ RSS Aggregator for blog posts about Django on Jython]
Line 3: Line 3:
[http://pipes.yahoo.com/pipes/pipe.info?_id=jroxOD1f3BGgvtiIqGIyXQ RSS Aggregator for blog posts about Django on Jython] == Summary of Django on Jython: The Gaps ==

The most important thing to know about Django on Jython is that we are almost there, and with clean code. End-to-end functionality is demonstrated by the admin tool running in full CRUD, along with a substantial number of unit tests. But this has been achieved by so far requiring only 6 lines of code in changes to Django trunk. (There will be more, however, see below.)

== Running on Jython ==
Line 7: Line 11:
 * Use the [https://jython.svn.sourceforge.net/svnroot/jython/branches/modern/ Modern branch] of Jython. This consolidated the bugs, workarounds, and patches found below -- plus a bunch more -- in a stable, almost-ready-to-be-merged-into-trunk version of Jython. The most important aspect is that we have tried to make Jython conform more to CPython, using Django as our guide, although there are some gaps - especially if Django already had incorporated fixes. Our driving goal is to converge on these gaps over time.  * Use the [https://jython.svn.sourceforge.net/svnroot/jython/branches/modern/ Modern branch] of Jython. This consolidated the bugs, workarounds, and patches found below -- plus a bunch more -- in a stable, almost-ready-to-be-merged-into-trunk version of Jython. The most important aspect is that we have tried to make Jython conform more to CPython, using Django as our guide, although there are some gaps - especially if Django already had incorporated fixes. Our driving goal is to converge on these gaps over time. Please note that is intended to be stable, performant code.
Line 15: Line 19:
 * Use the database backend [https://jython.svn.sourceforge.net/svnroot/jython/trunk/sandbox/jbaker/django/db/backends/postgresql_zxjdbc/ zxjdbc_postgresql], which was contributed by Leo Soto.  * Use the database backend [https://jython.svn.sourceforge.net/svnroot/jython/trunk/sandbox/jbaker/django/db/backends/postgresql_zxjdbc/ zxjdbc_postgresql], which was contributed by Leo Soto. Frank Wierzbicki has an experimental backend for !MySQL, this should be incorporated soon.

== Status ==
Line 23: Line 29:
 * Many doctests are fragile, because they depend on the dict traversal ordering; in Jython, this is different that CPython, and if we adopt {{{ConcurrentHashMap}}}, it's not even repeatable. This is a pervasive bug in Django.  * Many doctests are fragile, because they depend on the dict traversal ordering; in Jython, this is different that CPython, and if we adopt {{{ConcurrentHashMap}}}, it's not even repeatable. This would seem to be a pervasive bug in Django.
Line 33: Line 39:
---- == Next Steps ==

On the Django front, get more of the unit tests running!
Line 37: Line 45:
 * The test_extcall unit test currently fails. This appears to be a dependency on the dict traversal being the same. However, it's a mind bending test. The 2.3 version is particularly problematic because it's not modular at all. Google's GHOP has produced an improved version for Python 2.6 - we will look at this as a starting point.  * The test_extcall unit test currently fails. This appears to be a dependency on {{{dict traversal}}} being repeatable. However, it's a mind bending test. The 2.3 version is particularly problematic because it's not modular at all. Google's GHOP has just produced an improved version for Python 2.6 - we will look at this as a starting point.
Line 39: Line 47:
 * Tristan King provided a near complete subset of the functionality for {{{time.strptime}}}, as implemented in org.python.modules.time.Time. This needs to be enhanced. All unit tests in the CPythonLib version of test_time now pass except for {{{strptime}}} -- specifically the conversion specifier {{{'%c'}}} -- so we can also move to that, and discard our Jython version, when this is completed. That should be soon!  * Tristan King provided a near complete subset of the functionality for {{{time.strptime}}}, as implemented in {{{org.python.modules.time.Time}}}. This needs to be enhanced. All unit tests in the CPythonLib version of test_time now pass except for {{{strptime}}} -- specifically the conversion specifier {{{'%c'}}} -- so we can also move to that, and discard our Jython version, when this is completed. That should be soon!
Line 41: Line 49:
 * Decide whether we should use {{{ConcurentHashMap}}} or not as the backing hash map for {{{dict}}} and {{{__dict__}}}.  * Decide whether we should use {{{ConcurentHashMap}}} or not as the backing hash map for {{{dict}}} and {{{__dict__}}}. CHM introduces creation overhead, but it should prove to be far more scalable on multicore systems. The programming model is also far nicer with respect to Jython.
Line 43: Line 51:
=== Former Major problems === ----

== Historical Notes ==
Line 46: Line 56:

=== Formed Major Problems ===

[http://pipes.yahoo.com/pipes/pipe.info?_id=jroxOD1f3BGgvtiIqGIyXQ RSS Aggregator for blog posts about Django on Jython]

Summary of Django on Jython: The Gaps

The most important thing to know about Django on Jython is that we are almost there, and with clean code. End-to-end functionality is demonstrated by the admin tool running in full CRUD, along with a substantial number of unit tests. But this has been achieved by so far requiring only 6 lines of code in changes to Django trunk. (There will be more, however, see below.)

Running on Jython

To run Django on Jython, with a PostgreSQL backend, the following steps are necessary:

Status

Here's what works:

syncdb and the very cool Django admin run; many unit tests pass. You can run with internationalization enabled. You do need to run the dev server with --noreload for now. We need to document here how to run with [http://www.xhaus.com/modjy/ modjy], which is Alan Kennedy's servlet container for WSGI apps.

In running the model unit tests, here are the things we seem to be missing, accounting for most of the approximately 75 failures:

  • Many doctests are fragile, because they depend on the dict traversal ordering; in Jython, this is different that CPython, and if we adopt ConcurrentHashMap, it's not even repeatable. This would seem to be a pervasive bug in Django.

  • We still have some encoding problems, again seen in doctests. An example where output is expected to be lower case hex, not upper case. I fixed the problem in PyUnicode, but there are more places.

  • Problem with the ManagerDescriptor handling, in django.db.models.manager.

  • No decorators yet! (But they are coming soon, and are now available experimentally for Jython.)

There may be some other rough categories, we need to look at the failures more systematically. All that doctest noise is certainly annoying!

Next Steps

On the Django front, get more of the unit tests running!

Before we can push modern into trunk, the following needs to be done:

  • The test_extcall unit test currently fails. This appears to be a dependency on dict traversal being repeatable. However, it's a mind bending test. The 2.3 version is particularly problematic because it's not modular at all. Google's GHOP has just produced an improved version for Python 2.6 - we will look at this as a starting point.

  • Tristan King provided a near complete subset of the functionality for time.strptime, as implemented in org.python.modules.time.Time. This needs to be enhanced. All unit tests in the CPythonLib version of test_time now pass except for strptime -- specifically the conversion specifier '%c' -- so we can also move to that, and discard our Jython version, when this is completed. That should be soon!

  • Decide whether we should use ConcurentHashMap or not as the backing hash map for dict and __dict__. CHM introduces creation overhead, but it should prove to be far more scalable on multicore systems. The programming model is also far nicer with respect to Jython.

Historical Notes

(Preserved to show how much we had to go through...)

Formed Major Problems

Former Minor annoyances

Already Fixed

(And many more!)

DjangoOnJython (last edited 2014-05-22 23:24:19 by AdamBurke)