|
Size: 5422
Comment: Bringing back rescued text
|
Size: 5417
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 62: | Line 62: |
| /* Constants taken from CPython object.h in v3.3 */ |
// Constants taken from CPython object.h in v3.3 |
Buffer Protocol
This page proposes (now begins to document) a design for a Jython equivalent to the CPython buffer protocol. A good place to start is this worked example (pdf) that motivated the current design.
Contents
Notes from a User Perspective
What is the Buffer API?
The Jython Buffer API is an interface you can use in Java when accessing or implementing certain built-in types, or your own. It is the basis of the type memoryview with which it shares many features.
In CPython, certain objects are based on an underlying memory array or buffer. The CPython designers judged it desirable to be able to access that buffer directly, without intermediate copying. CPython provides this at the C level in the form of the buffer protocol. This is used heaviliy in the implementation of some core types and standard library modules.
The capability is just as useful in Jython, and has been implemented for version 2.7.
Accessing an Object that has the Buffer API
Objects flag their willingness to provide a buffer by implementing the following interface:
The flags argument indicates the buffer characteristics wanted by the consumer, and what kind of data organisation the consumer can cope with. The return is an implementation of PyBuffer appropriate to the storage organisation of the exporter. This interface is quite rich, but the Javadoc is thorough. In fact, the interface is defined in two stages: a part that is independent of the type of unit in the buffer, and a part that is definite that the units are bytes.
The type-agnostic part of the interface is PyBUF:
1 public interface PyBUF {
2
3 boolean isReadonly();
4
5 // Access to buffer (client responsible for indexing)
6 int getNdim();
7 int[] getShape();
8 int getItemsize();
9 int getLen();
10 int[] getStrides();
11 int[] getSuboffsets();
12 boolean isContiguous(char order);
13
14 // Constants taken from CPython object.h in v3.3
15 static final int MAX_NDIM = 64;
16 static final int WRITABLE = 0x0001;
17 static final int SIMPLE = 0;
18 static final int FORMAT = 0x0004;
19 static final int ND = 0x0008;
20 static final int STRIDES = 0x0010 | ND;
21 static final int C_CONTIGUOUS = 0x0020 | STRIDES;
22 static final int F_CONTIGUOUS = 0x0040 | STRIDES;
23 static final int ANY_CONTIGUOUS = 0x0080 | STRIDES;
24 static final int INDIRECT = 0x0100 | STRIDES;
25 static final int CONTIG = ND | WRITABLE;
26 static final int CONTIG_RO = ND;
27 static final int STRIDED = STRIDES | WRITABLE;
28 static final int STRIDED_RO = STRIDES;
29 static final int RECORDS = STRIDES | WRITABLE | FORMAT;
30 static final int RECORDS_RO = STRIDES | FORMAT;
31 static final int FULL = INDIRECT | WRITABLE | FORMAT;
32 static final int FULL_RO = INDIRECT | FORMAT;
33 static final int NAVIGATION = SIMPLE | ND | STRIDES | INDIRECT;
34 static final int IS_C_CONTIGUOUS = C_CONTIGUOUS & ~STRIDES;
35 static final int IS_F_CONTIGUOUS = F_CONTIGUOUS & ~STRIDES;
36 static final int CONTIGUITY = (C_CONTIGUOUS | F_CONTIGUOUS | ANY_CONTIGUOUS) & ~STRIDES;
37 }
The byte-oriented part of the interface is PyBuffer. For the most part, you can ignore the difference between PyBUF and PyBuffer and think of all the methods as belonging to PyBuffer:
1 public interface PyBuffer extends PyBUF, BufferProtocol {
2
3 // Access to buffer contents (index calculation done by buffer)
4 byte byteAt(int index) throws IndexOutOfBoundsException;
5 int intAt(int index) throws IndexOutOfBoundsException;
6 void storeAt(byte value, int index) throws IndexOutOfBoundsException;
7 byte byteAt(int... indices) throws IndexOutOfBoundsException;
8 int intAt(int... indices) throws IndexOutOfBoundsException;
9 void storeAt(byte value, int... indices) throws IndexOutOfBoundsException;
10
11 // Bulk operations (index calculation done by buffer)
12 void copyTo(byte[] dest, int destPos);
13 void copyTo(int srcIndex, byte[] dest, int destPos, int length);
14 void copyFrom(byte[] src, int srcPos, int destIndex, int length);
15 void copyFrom(PyBuffer src);
16
17 // Releasing a buffer or getting another (or a slice)
18 void release();
19 boolean isReleased();
20 @Override PyBuffer getBuffer(int flags); // from BufferProtocol
21 public PyBuffer getBufferSlice(int flags, int start, int length);
22 public PyBuffer getBufferSlice(int flags, int start, int length, int stride);
23
24 // Access to buffer (client responsible for indexing)
25 public static class Pointer {
26 public byte[] storage;
27 public int offset;
28 public Pointer(byte[] storage, int offset) {
29 this.storage = storage;
30 this.offset = offset;
31 }
32 }
33
34 PyBuffer.Pointer getBuf();
35 PyBuffer.Pointer getPointer(int index);
36 PyBuffer.Pointer getPointer(int... indices);
37
38 // Interpreting the bytes
39 String getFormat();
40 }
Notice that an object that implements PyBuffer must itself implement the BufferProtocol. A buffer can give you a buffer, which may just be itself or could be an independent object, depending on the implementation.