How can you bind exceptions with custom fields and constructors in pybind11 and still have them function as python exception?

This appears to be a known limitation in pybind11. I read through all the docs, whatever bug reports seemed applicable, and everything I could find in the pybind11 gitter. I have a custom exception class in c++ that contains custom constructors and fields. A very basic example of such a class, trimmed for space is here:

class BadData : public std::exception
{
  public:
    // Constructors
    BadData()
      : msg(),
        stack(),
        _name("BadData")
    {}

    BadData(std::string _msg, std::string _stack)
      : msg(_msg),
        stack(_stack),
        _name("BadData")
    {}

    const std::string&
    getMsg() const
    {
      return msg;
    }

    void
    setMsg(const std::string& arg)
    {
      msg = arg;
    }

    // Member stack
    const std::string&
    getStack() const
    {
      return stack;
    }

    void
    setStack(const std::string& arg)
    {
      stack = arg;
    }
  private:
    std::string msg;
    std::string stack;
    std::string _name;

I currently have python binding code that binds this into python, but it is custom generated and we'd much rather use pybind11 due to its simplicity and compile speed.

The default mechanism for binding an exception into pybind11 would look like

py::register_exception<BadData>(module, "BadData");

That will create an automatic translation between the C++ exception and the python exception, with the what() value of the c++ exception translating into the message of the python exception. However, all the extra data from the c++ exception is lost and if you're trying to throw the exception in python and catch it in c++, you cannot throw it with any of the extra data.

You can bind extra data onto the python object using the attr and I even went somewhat down the path of trying to extend the pybind11:exception class to make it easier to add custom fields to exceptions.

  template <typename type>
  class exception11 : public ::py::exception<type>
  {
   public:
    exception11(::py::handle scope, const char *name, PyObject *base = PyExc_Exception)
      : ::py::exception<type>(scope, name, base)
    {}

    template <typename Func, typename... Extra>
    exception11 &def(const char *name_, Func&& f, const Extra&... extra) {
      ::py::cpp_function cf(::py::method_adaptor<type>(std::forward<Func>(f)),
                            ::py::name(name_),
                            ::py::is_method(*this),
                            ::py::sibling(getattr(*this, name_, ::py::none())),
                            extra...);
      this->attr(cf.name()) = cf;
      return *this;
    }
  };

This adds a def function to exceptions similar to what is done with class_. The naive approach to using this doesn't work

    exception11< ::example::data::BadData>(module, "BadData")
      .def("getStack", &::example::data::BadData::getStack);

Because there is no automatic translation between BadData in c++ and in python. You can try to work around this by binding in a lambda:

    .def("getStack", [](py::object& obj) {
      ::example::data::BadData *cls = obj.cast< ::example::data::BadData* >();
      return cls->getStack();
    });

The obj.cast there also fails because there is no automatic conversion. Basically, with no place to store the c++ instance, there isn't really a workable solution for this approach that I could find. In addition I couldn't find a way to bind in custom constructors at all, which made usability on python very weak.

The next attempt was based on a suggestion in the pybind11 that you could use the python exception type as a metaclass a normal class_ and have python recognize it as a valid exception. I tried a plethora of variations on this approach.

py::class_< ::example::data::BadData >(module, "BadData", py::dynamic_attr(), py::reinterpret_borrow<py::object>(PyExc_Exception))
py::class_< ::example::data::BadData >(module, "BadData", py::dynamic_attr(), py::cast(PyExc_Exception))
py::class_< ::example::data::BadData >(module, "BadData", py::dynamic_attr(), py::cast(PyExc_Exception->ob_type))
py::class_< ::example::data::BadData>(module, "BadData", py::metaclass((PyObject *) &PyExc_Exception->ob_type))

There were more that I don't have saved. But the overall results was either 1) It was ignored completely or 2) it failed to compile or 3) It compiled and then immediately segfaulted or ImportError'd when trying to make an instance. There might have been one that segfaulted on module import too. It all blurs together. Maybe there is some magic formula that would make such a thing work, but I was unable to find it. From my reading of the pybind11 internals, I do not believe that such a thing is actually possible. Inheriting from a raw python type does not seem to be something it is setup to let you do.

The last thing I tried seemed really clever. I made a python exception type

  static py::exception<::example::data::BadData> exc_BadData(module, "BadDataBase");

and then had my pybind11 class_ inherit from that.

  py::class_< ::example::data::BadData >(module, "BadData", py::dynamic_attr(), exc_BadData)

But that also segfaulted on import too. So I'm basically back to square one with this.