Differences between revisions 28 and 29
Revision 28 as of 2005-04-06 12:57:18
Size: 12631
Editor: zigzag
Comment: Replace uncompilable example of exposing mutable C++ class with the working one.
Revision 29 as of 2005-10-31 00:42:55
Size: 12657
Editor: 68-189-248-26
Comment: Fixed non-WikiWords.
Deletions are marked like this. Additions are marked like this.
Line 252: Line 252:
The method defined upper does not work if part of your object is written in Python. The PyObject* associated to the object may still be delete by Python. If you want to ensure the lifetime of the object, you have to increase manually the reference couting in the C++ Wrapper AND use the above method. With this, you'll be sure the PyObject* is not destroyed while the C++ still exists.

So, write the MyClassWrap as : {{{
The method defined upper does not work if part of your object is written in Python. The !PyObject* associated to the object may still be delete by Python. If you want to ensure the lifetime of the object, you have to increase manually the reference couting in the C++ Wrapper ''and'' use the above method. With this, you'll be sure the !PyObject* is not destroyed while the C++ still exists.

So, write the !MyClassWrap as : {{{
Line 281: Line 281:
The problem is, if the dynamic type of my_obj is not MyClass but a class derived from MyClass, you won't have, in Python, the interface of the derived class. If you want to create the object with the interface of the real dynamic type you have to use the ResultConverter created by Boost : {{{ The problem is, if the dynamic type of my_obj is not !MyClass but a class derived from !MyClass, you won't have, in Python, the interface of the derived class. If you want to create the object with the interface of the real dynamic type you have to use the !ResultConverter created by Boost : {{{
Line 293: Line 293:
Now, if you can copy the object and use the copy instead of a reference, you can change the ResultConverter and use {{{return_by_value}}} instead of {{{reference_existing_object}}}. Now, if you can copy the object and use the copy instead of a reference, you can change the !ResultConverter and use {{{return_by_value}}} instead of {{{reference_existing_object}}}.
Line 314: Line 314:
Boost will create a new MyClass wrapper and you won't have access to the Python methods. To access the Python object you have to explicitly send the Python wrapper boost gave you at the object creation :{{{ Boost will create a new !MyClass wrapper and you won't have access to the Python methods. To access the Python object you have to explicitly send the Python wrapper boost gave you at the object creation :{{{
Line 331: Line 331:
Of course, if you also have C++ classes derived from MyClass, you'll have to implement the function as :{{{ Of course, if you also have C++ classes derived from !MyClass, you'll have to implement the function as :{{{
Line 370: Line 370:
=== [NOTE] Problems using M$ Visual C++ Debugger ===

If you want to debug your extensions using the M$ Visual C++ debugger you need to remember to compile all libraries and extensions with the '''/MDd''' flag.
=== [NOTE] Problems using MS Visual C++ Debugger ===

If you want to debug your extensions using the MS Visual C++ debugger you need to remember to compile all libraries and extensions with the '''/MDd''' flag.
Line 376: Line 376:
''HEAP[python.exe]: Invalid Address specified to RtlValidateHeap( 00E30000, 00D48FA8 )'' {{{HEAP[python.exe]: Invalid Address specified to RtlValidateHeap( 00E30000, 00D48FA8 )}}}
Line 381: Line 381:
AFAIK, the actual problem is, if you've not compiled all your libs/extensions with ''/MDd'', that you have objects on different heaps and the debugger stops with an assert because the extensions tries to access objects from the wrong heap. As far as I know, the actual problem is, if you've not compiled all your libs/extensions with ''/MDd'', that you have objects on different heaps and the debugger stops with an assert because the extensions tries to access objects from the wrong heap.

How to expose...

TableOfContents

static class data members

object x_class 
    = class_<X>("X") 
         .def( ... ) 
         ... 
         ; 
 
x_class.attr("fu") = X::fu; 
x_class.attr("bar") = X::bar; 
... 

Since version 1.30 you can use class_ method:

   .add_static_property("name", &fget [,&fset])

static class functions

It's likely to be in 1.30 release.

Meanwhile you should do something which mirrors pure Python:

class foo(object):
    def f(x,y):
        return x*y
    f = staticmethod(f)

So in C++, it would be something like:

    class_<foo>("foo")
        .def("f", &foo::f)
        .staticmethod("f")  // **
        ;

Where the marked line would be implemented something like this:

self& staticmethod(char const* name)
{
    dict d(handle<>(borrowed(downcast<PyTypeObject>(this->ptr())->tp_dict))); 
    object method = (object)(d[name]); 
    this->attr(name) = object(handle<>(
         PyStaticMethod_New( callable_check(method.ptr()) ) 
      ));
    return *this;
}

To create overloaded staticmethods in python, you overload first, then you make it static.

module level objects

at module creation time

First, create those objects like

object class_X = class_<X>("X");
object x = class_X();

Second, expose them:

scope().attr("x") = x; // injects x into current scope

By default current scope is module.

at run-time

Use a function:

template <class T>
void set(const std::string& name, const T& value) {
  interpreter()->mainmodule()[name] = value;
}

Note:: interpreter() is going to be added.

mutable C++ object

Perhaps you'd like the resulting Python object to contain a raw pointer to the argument? In that case, the caveat is that if the lifetime of the C++ object ends before that of the Python object, that pointer will dangle and using the Python object may cause a crash.

Here's how to expose mutable C++ object during module initialisation:

  scope().attr("a") = object(ptr(&class_instance));

std::C++ container

You can always wrap the container with class_ directive. For example for std::map:

template<class Key, class Val>
struct map_item
{
    typedef std::map<Key,Val> Map;

    static Val& get(Map const& self, const Key idx) {
      if( self.find(idx) != self.end() ) return self[idx];
      PyErr_SetString(PyExc_KeyError,"Map key not found");
      throw_error_already_set();
    }

    static void set(Map& self, const Key idx, const Val val) { self[idx]=val; }

    static void del(Map& self, const Key n) { self.erase(n); }

    static bool in(Map const& self, const Key n) { return self.find(n) != self.end(); }

    static list keys(Map const& self)
    {
        list t;
        for(Map::const_iterator it=self.begin(); it!=self.end(); ++it)
            t.append(it->first);
        return t;
    }
    static list values(Map const& self)
    {
        list t;
        for(Map::const_iterator it=self.begin(); it!=self.end(); ++it)
            t.append(it->second);
        return t;
    }
    static list items(Map const& self)
    {
        list t;
        for(Map::const_iterator it=self.begin(); it!=self.end(); ++it)
            t.append( make_tuple(it->first, it->second) );
        return t;
    }
}

using namespace boost::python;
typedef std::map<Key,Val> Map;
class_<Map>("Map")
  .def("__len__", &Map::size)
  .def("__getitem__", &map_item<Key,Val>().get, return_value_policy<copy_non_const_reference>() )
  .def("__setitem__", &map_item<Key,Val>().set)
  .def("__delitem__", &map_item<Key,Val>().del)
  .def("clear", &Map::clear)
  .def("__contains__", &map_item<Key,Val>().in)
  .def("has_key", &map_item<Key,Val>().in)
  .def("keys", &map_item<Key,Val>().keys)
  .def("values", &map_item<Key,Val>().values)
  .def("items", &map_item<Key,Val>().items)
  ;

"Raw" function

I want to write in python:

  •     def function( *args ):
            # Mess arount with elements of args.

You can make one with the library's implementation details. Be warned that these interfaces may change, but hopefully we'll have an "official" interface for what you want to do by then.

python::objects::function_object(
    f                       // must be py_function compatible
    , 0, std::numeric_limits<unsigned>::max()  // arity range
    , python::detail::keyword_range());        // no keywords

will create a Python callable object.

f is any function pointer, function reference, or function object which can be invoked with two PyObject* arguments and returns something convertible to a PyObject*.

You can add this to your module namespace with:

scope().attr("name") = function_object(f, ... );

Now you can also

class_<foo>("foo")
    .def("bar",function_object(f, ... ) );

aswell, but not

function_object bar(f, ... );
...
class_<foo>("foo")
    .def("bar",bar);

because function_object is a function, not a class.

getter and setter methods as a property

Suppose you have class "C" with methods "getA" and "setA" and you want to expose them as a property "a". It's not a problem with  .add_property("a", &C::getA, &C::setA) unless you need to assign ../CallPolicy to them. In that case use "make_function": {{{ .add_property("a",

  • make_function(&C::getA, return_value_policy<...>()), make_function(&C::setA, with_custodian_and_ward<...>())}}}

How to get...

C++ object from Python

If you have a reference, use handle:

handle<> x(whatever);   // new reference
handle<> x(borrowed(whatever));  // borrowed reference

object(x); // now we have an object.

Than you can try to extract a value:

Type v = extract<Type>(x);

Or from the start. Suppouse you wrap your class T as:

object Tclass = class_<T>("T")/* defs, etc. */;

Then you can create such object by

object pyt = Tclass(/* constructor args*/);

and get the underlying T this way:

T& t = extract<T&>(pyt);

See ["../extract"]

multithreading support for my function

The best way to safely unblock Python threads is to write a thin wrapper around your function which uses Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS around a call to the real function.

ownership of C++ object

Wrap object with auto_ptr<> storage:

class_<Myclass, auto_ptr<MyClass> >("MyClass");

Write a small helper function to call the function which wants to take ownership of MyClass object:

void caller(auto_ptr<MyClass> obj)
{
  new_owner_function(obj.get());
  obj.release();
}

Wrap caller under the new_owner_function Python name.

See also http://www.boost.org/libs/python/doc/v2/faq.html#ownership

ownership of C++ object extended in Python

The method defined upper does not work if part of your object is written in Python. The PyObject* associated to the object may still be delete by Python. If you want to ensure the lifetime of the object, you have to increase manually the reference couting in the C++ Wrapper and use the above method. With this, you'll be sure the PyObject* is not destroyed while the C++ still exists.

So, write the MyClassWrap as :

class MyClassWrap : public MyClass
{
  MyClassWrap( PyObject* self_) : self(self_) { PyINCREF(self); }
  MyClassWrap( PyObject* self_, const MyClass& copy ) : MyClass(copy), self(self_) { PyINCREF(self); }
  ~MyClassWrap() { PyDECREF(self); }
  ...
  PyObject *self;
};

And in the module definition :

class_<MyClass, auto_ptr<MyClassWrap> >("MyClass");
implicitly_convertible<auto_ptr<MyClassWrap>, auto_ptr<MyClass> >();

And you still write the thin wrapper as described above.

python object from derived C++ object

The first, but incomplete, way to do this is to create an object with :

void MyFct( MyClass* my_obj )
{
  object obj(my_obj);
  python_fct(obj);
}

The problem is, if the dynamic type of my_obj is not MyClass but a class derived from MyClass, you won't have, in Python, the interface of the derived class. If you want to create the object with the interface of the real dynamic type you have to use the ResultConverter created by Boost :

void MyFct( MyClass* my_obj )
{
  reference_existing_object::apply<MyClass*>::type converter;
  PyObject* obj = converter( my_obj );
  object real_obj = object( handle<>( obj ) );
  python_fct(real_obj);
}

The python object must not be keep outside the execution of python_fct. If you do, then you have to take extra care about the lifetime of the Python object and the underlying C++ object for there are no lifetime guards possible (the original object being a C++ one, you have no automatic way to tie the lifetime of this object with the lifetime of the Python wrapper).

Now, if you can copy the object and use the copy instead of a reference, you can change the ResultConverter and use return_by_value instead of reference_existing_object.

python object from extended C++ object

We still want the previous behaviour, but now the C++ class may be extended in Python. We so have a class wrapper :

class MyClassWrap : public MyClass
{
public:
  MyClassWrap(PyObject *self_, ...) : MyClass(...), self(self_) { ... }
  virtual bool virtual_method() { return call_method<bool>(self, "virtual_method"); }
  ...
  PyObject *self;
  ...
};

Let's say you have a C++ method, calling Python with the object. If you create the function as :

bool my_fct(MyClass* my_obj)
{
  bool result = extract<bool>( python_function(my_obj) );
}

Boost will create a new MyClass wrapper and you won't have access to the Python methods. To access the Python object you have to explicitly send the Python wrapper boost gave you at the object creation :

bool my_fct(MyClass* my_obj)
{
  MyClassWrap *wrap_obj = dynamic_cast<MyClassWrap*>(my_obj);
  object final_obj;
  if( wrap_obj != NULL ) // Means we have a Python-defined object
    {
      final_obj = object(handle<>(borrowed(wrap_obj->self)));
    }
  else
    {
      final_obj = my_obj;
    }
  return extract<bool>( python_function(final_obj) );
}

Of course, if you also have C++ classes derived from MyClass, you'll have to implement the function as :

bool my_fct(MyClass* my_obj)
{
  MyClassWrap *wrap_obj = dynamic_cast<MyClassWrap*>(my_obj);
  object final_obj;
  if( wrap_obj != NULL ) // Means we have a Python-defined object
    {
      final_obj = object(handle<>(borrowed(wrap_obj->self)));
    }
  else
    {
      reference_existing_object::apply<const Base*>::type converter;
      final_obj = object(handle<>(convert(my_obj)));
    }
  return extract<bool>( python_function(final_obj) );
}

access to a Python extension in the same app that embeds Python?

Suppose you have an extension module "hello" with function "greet".

Module init function named "inithello" must be imported:

  PyImport_AppendInittab("hello", inithello);

before

  Py_Initialize();

and later:

  object strRes(handle<>(PyRun_String(
                "from hello import *\n"
                "result = greet()\n",
                Py_file_input,
                main_namespace.ptr(),
                main_namespace.ptr()) ));

  const char* res = extract<const char*>(main_namespace["result"]);

Howto debug your extensions..

[NOTE] Problems using MS Visual C++ Debugger

If you want to debug your extensions using the MS Visual C++ debugger you need to remember to compile all libraries and extensions with the /MDd flag.

If you forget this compiler flag, it's likely that your debugger will halt on a breakpoint with an error something like this:

HEAP[python.exe]: Invalid Address specified to RtlValidateHeap( 00E30000, 00D48FA8 )

The actual statement which halts your debugger is executed in the NTDLL and therefor is not debuggable. While this seems to apply to all of the boost libraries, it can be annoying and it's hard to actually find the problem.

As far as I know, the actual problem is, if you've not compiled all your libs/extensions with /MDd, that you have objects on different heaps and the debugger stops with an assert because the extensions tries to access objects from the wrong heap.

boost.python/HowTo (last edited 2015-04-21 17:00:36 by HansDembinski)

Unable to edit the page? See the FrontPage for instructions.