|
Size: 2299
Comment:
|
Size: 2292
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 5: | Line 5: |
| Overview ============ |
= Overview = |
| Line 8: | Line 7: |
| The buffer protocol allows different Python types to exchange a pointer to a sequence of internal buffers. This functionality is *extremely* useful for sharing large segments of memory between different high-level objects, but it's too limited and has issues. | The buffer protocol allows different Python types to exchange a pointer to a sequence of internal buffers. This functionality is '''extremely''' useful for sharing large segments of memory between different high-level objects, but it's too limited and has issues. |
| Line 10: | Line 9: |
| 1) There is the little used "sequence-of-segments" option. 2) There is no way for a consumer to tell the protocol-exporting object it is "finished" with its view of the memory and therefore no way for the object to be sure that it can reallocate the pointer to the memory that it owns (the array object reallocating its memory after sharing it with the buffer object led to the infamous buffer-object problem). 3) Memory is just a pointer. There is no way to describe what's "in" the memory (float, int, C-structure, etc.) 4) There is no shape information provided for the memory. But, several array-like Python types could make use of a standard way to describe the shape of the memory (wxPython, GTK, CVXOPT, PyVox, Audio and Video Libraries, ctypes, NumPy) |
1. There is the little (never?) used "sequence-of-segments" option. 2. There is no way for a consumer to tell the protocol-exporting object it is "finished" with its view of the memory and therefore no way for the object to be sure that it can reallocate the pointer to the memory that it owns (the array object reallocating its memory after sharing it with the buffer object which held the original pointer led to the infamous buffer-object problem). 3. Memory is just a pointer. There is no way to describe what's "in" the memory (float, int, C-structure, etc.) 4. There is no shape information provided for the memory. But, several array-like Python types could make use of a standard way to describe the shape-interpretation of the memory (!wxPython, GTK, CVXOPT, !PyVox, Audio and Video Libraries, ctypes, !NumPy) |
| Line 15: | Line 14: |
| Proposal ========= |
= Proposal = |
| Line 18: | Line 16: |
| 1) Replace the buffer protocol that allows sharing of a single pointer to memory 2) Have the protocol define a way to describe what's in the memory location (this should unify what is done now in struct, array, ctypes, and NumPy) 3) Have the protocol be able to share information about shape (and striding if any) 4) Allow exporting objects to define some function that should be called when the consumer object is "done" with the view. |
1. Get rid of the char-buffer and multiple-segment sections of the buffer-protocol. 2. Add a new function to the protocol that should be called when the consumer object is "done" with the view. 3. Add a new function to allow the protocol to describe what is in memory (unifying what is currently done now in struct and array) 4. Add a new function to allow the protocol to share shape information |
| Line 23: | Line 21: |
| Idea ====== |
= Idea = |
| Line 33: | Line 30: |
| Details =========== |
= Details = |
| Line 37: | Line 33: |
| Questions =========== |
= Questions = |
| Line 41: | Line 36: |
| Problems =========== |
= Problems = |
This pre-PEP proposes enhancing the buffer protocol in Python 3000 to implement the array interface (protocol).
This Wiki will serve as a place to record import points raised during the mailing list discussion.
Overview
The buffer protocol allows different Python types to exchange a pointer to a sequence of internal buffers. This functionality is extremely useful for sharing large segments of memory between different high-level objects, but it's too limited and has issues.
- There is the little (never?) used "sequence-of-segments" option.
- There is no way for a consumer to tell the protocol-exporting object it is "finished" with its view of the memory and therefore no way for the object to be sure that it can reallocate the pointer to the memory that it owns (the array object reallocating its memory after sharing it with the buffer object which held the original pointer led to the infamous buffer-object problem).
- Memory is just a pointer. There is no way to describe what's "in" the memory (float, int, C-structure, etc.)
There is no shape information provided for the memory. But, several array-like Python types could make use of a standard way to describe the shape-interpretation of the memory (!wxPython, GTK, CVXOPT, PyVox, Audio and Video Libraries, ctypes, NumPy)
Proposal
- Get rid of the char-buffer and multiple-segment sections of the buffer-protocol.
- Add a new function to the protocol that should be called when the consumer object is "done" with the view.
- Add a new function to allow the protocol to describe what is in memory (unifying what is currently done now in struct and array)
- Add a new function to allow the protocol to share shape information
Idea
All that is needed is to create a Python "memory_view" object that can contain all the information needed and be returned when the buffer protocol is called --- when it is garbage-collected, the "bp_release_view" function is called on the exporting object.
This "memory_view" is essentially the old Numeric C-structure (including the fact that the data-format is described by another C-structure).
This object is what the buffer protocol should return.
Details
Questions