----------------------------
The 2.0.0 release of nanobind is entirely dedicated to *types* [f1]_! The
project has always advertised seamless Python ↔ C++ interoperability, and this
release tries to bring a similar level of interoperability to static type
checkers like `MyPy <https://github.com/python/mypy>`__, `PyRight
<https://github.com/microsoft/pyright>`__, `PyType
<https://github.com/google/pytype>`__, and editors with interactive
autocompletion like `Visual Studio Code <https://code.visualstudio.com>`__,
`PyCharm <https://www.jetbrains.com/pycharm/>`__, and many other `LSP
<https://en.wikipedia.org/wiki/Language_Server_Protocol>`__-compatible IDEs.
This required work on three fronts:
1. **Stub generation**: the above tools all analyze Python code statically
without running it. Because the import mechanism of compiled extensions
depends the Python interpreter, these tools weren't able to inspect the
contents of nanobind-based extensions.
The usual solution involves writing `stubs
<https://typing.readthedocs.io/en/latest/source/stubs.html>`__ that expose
the module contents to static analysis tools. However, writing stubs by hand
is tedious and error-prone.
This release adds tooling to automatically extract stubs from existing
extensions. The process is fully integrated into the CMake-based build
system and explained in a :ref:`new documentation section <stubs>`.
2. **Better default annotations**: once stubs were available, this revealed the
next problem: the default nanobind-provided function and class signatures
were too rudimentary, and this led to a user poor experience.
The release therefore improves many builtin type caster so that they produce
more accurate type signatures. For example, the STL ``std::vector<T>``
caster now renders as ``collections.abc.Sequence[T]`` in stubs when it is
used as an *input*, and ``list[T]`` when it is used as part of a return
value. The :cpp:func:`nb::make_*_iterator() <make_iterator>` family of
functions return typed iterators, etc.
3. **Advanced customization**: a subset of the type signatures in larger
binding projects will generally require further customization. The features
listed below aim to enable precisely this:
* In Python, many built-in types are *generic* and can be *parameterized* (e.g.,
``list[int]``). The :cpp:class:`nb::typed\<T, Ts...\> <typed>` wrapper
enables such parameterization within C++ (for example, the
``int``-specialized list would be written as ``nb::typed<nb::list,
int>``). :ref:`Read more <typing_generics_parameterizing>`.
* The opposite is also possible: passing :cpp:class:`nb::is_generic()
<is_generic>` to the class binding constructor
.. code-block:: cpp
nb::class_<MyType>(m, "MyType", nb::is_generic())
produces a *generic* type that can be parameterized in Python (e.g.
``MyType[int]``). :ref:`Read more <typing_generics_creating>`.
* The :cpp:class:`nb::sig <sig>` annotation overrides the
signature of a function or method, e.g.:
.. code-block:: cpp
m.def("f", &f, nb::sig("def f(x: Foo = Foo(0)) -> None"), "docstring");
Each binding of an overloaded function can be customized separately. This
feature can be used to add decorators or control how default arguments are
rendered. :ref:`Read more <typing_signature_functions>`.
* The :cpp:class:`nb::sig <sig>` annotation can also override *class
signatures* in generated stubs. Stubs often take certain liberties in
deviating somewhat from the precise type signature of the underlying
implementation. For example, the following annotation adds an abstract
base class advertising that the class implements a typed iterator.
.. code-block:: cpp
using IntVec = std::vector<int>;
nb::class_<IntVec>(m, "IntVec",
nb::sig("class IntVec(collections.abc.Iterable[int])"));
Nanobind can't subclass Python types, hence this declaration is
technically untrue. On the flipside, such a declaration can assist static
checkers and improve auto-completion in visual IDEs. This is fine since
these tools only perform a static analysis and never import the actual
extension. :ref:`Read more <typing_signature_classes>`.
* The :cpp:struct:`nb::for_setter <for_setter>` and
:cpp:struct:`nb::for_getter <for_getter>` annotations enable passing
function binding annotations (e.g., signature overrides) specifically to
the setter or the getter part of a property.
* The :cpp:class:`nb::arg("name") <arg>` argument annotation (and
``"name"_a`` shorthand) now have a :cpp:func:`.sig("signature")
<arg::sig>` member to control how a default value is rendered in the stubs
and docstrings. This provides more targeted control compared to overriding
the entire function signature.
* Finally, nanobind's stub generator supports :ref:`pattern files
<pattern_files>` containing custom stub replacement rules. This catch-all
solution addresses the needs of advanced binding projects, for which the
above list of features may still not be sufficient.
Most importantly, it was possible to support these improvements with minimal
changes to the core parts of nanobind.
These release breaks API and ABI compatibility, requiring a new major version
according to `SemVer <http://semver.org>`__. The following changes are
noteworthy:
* The :cpp:class:`nb::enum_\<T\>() <enum_>` binding declaration is now a
wrapper that creates either a ``enum.Enum`` or ``enum.IntEnum``-derived type.
Previously, nanobind relied on a custom enumeration base class that was a
frequent source of friction for users.
This change may break code that casts entries to integers, which now only
works for arithmetic (``enum.IntEnum``-derived) enumerations. Replace
``int(my_enum_entry)`` with ``my_enum_entry.value`` to work around the issue.
* The :cpp:func:`nb::bind_vector\<T\>() <bind_vector>` and
:cpp:func:`nb::bind_map\<T\>() <bind_map>` interfaces were found to be
severely flawed since element access (``__getitem__``) created views into the
internal state of the STL type that were not stable across subsequent
modifications.
This could lead to unexpected changes to array elements and undefined
behavior when the underlying storage was reallocated (i.e., use-after-free).
nanobind 2.0.0 improves these types so that they are safe to use, but this
means that element access must now copy by default, potentially making them
less convenient. The documentation of :cpp:func:`nb::bind_vector\<T\>()
<bind_vector>` discusses the issue at length and presents alternative
solutions.
* The functions :cpp:func:`nb::make_iterator() <make_iterator>`,
:cpp:func:`nb::make_value_iterator() <make_value_iterator>` and
:cpp:func:`nb::make_key_iterator() <make_key_iterator>` suffer from the same
issue as :cpp:func:`nb::bind_vector() <bind_vector>` explained above.
nanobind 2.0.0 improves these operations so that they are safe to use, but
this means that iterator access must now copy by default, potentially making
them less convenient. The documentation of :cpp:func:`nb::make_iterator()
<make_iterator>` discusses the issue and presents alternative solutions.
* The ``nb::raw_doc`` annotation was found to be too inflexible and was
removed in this version.
* The ``nb::typed`` wrapper listed above actually already existed in previous
nanobind versions but was awkward to use, as it required the user to provide
a custom type formatter. This release makes the interface more convenient.
* The ``nb::any`` placeholder to specify an unconstrained
:cpp:class:`nb::ndarray <ndarray>` axis was removed. This name was given to a
new wrapper type :cpp:class:`nb::any` indicating ``typing.Any``-typed
values.
All use of ``nb::any`` in existing code must be replaced with ``-1`` (for
example, ``nb::shape<3, nb::any, 4>`` → ``nb::shape<3, -1, 4>``).
* :ref:`Keyword-only arguments <kw_only>` are now supported, and can be
indicated using the new :cpp:struct:`nb::kw_only() <kw_only>` function
annotation. (PR `448 <https://github.com/wjakob/nanobind/pull/448>`__).
* nanobind classes now permit overriding ``__new__``, in order to
support C++ singletons, caches, and other types that expose factory
functions rather than ordinary constructors. Read the section on
:ref:`customizing Python object creation <custom_new>` for more details.
(PR `473 <https://github.com/wjakob/nanobind/pull/473>`__).
* When binding methods on a class ``T``, nanobind will now produce a Python
function that expects a self argument of type ``T``. Previously, it would
use the type of the member pointer to determine the Python function
signature, which could be a base of ``T``, which would create problems
if nanobind did not know about that base.
(PR `471 <https://github.com/wjakob/nanobind/pull/471>`__).
* nanobind can now handle keyword arguments that are not interned, which avoids
spurious ``TypeError`` exceptions in constructs like
``fn(**pickle.loads(...))``. The speed of normal function calls (which
generally do have interned keyword arguments) should be unaffected. (PR `469
<https://github.com/wjakob/nanobind/pull/469>`__).
* The ``owner=nb::handle()`` default value of the :cpp:func:`nb::ndarray
<ndarray>` constructor was removed since it was bug-prone. You now have to
specify the owner explicitly. The previous default (``nb::handle()``)
continues to be a valid argument.
* There have been some changes to the API for type casters in order to
avoid undefined behavior in certain cases. (PR `549
<https://github.com/wjakob/nanobind/pull/549>`__).
* Type casters that implement custom cast operators must now define a
member function template ``can_cast<T>()``, which returns false if
``operator cast_t<T>()`` would raise an exception and true otherwise.
``can_cast<T>()`` will be called only after a successful call to
``from_python()``, and might not be called at all if the caller of
``operator cast_t<T>()`` can cope with a raised exception.
(Users of the ``NB_TYPE_CASTER()`` convenience macro need not worry
about this; it produces cast operators that never raise exceptions,
and therefore provides a ``can_cast<T>()`` that always returns true.)
* Many type casters for container types (``std::vector<T>``,
``std::optional<T>``, etc) implement their ``from_python()`` methods
by delegating to another, "inner" type caster (``T`` in these examples)
that is allocated on the stack inside ``from_python()``. Container casters
implemented in this way should make two changes in order to take advantage
of the new safety features:
* Wrap your ``flags`` (received as an argument of the outer caster's
``from_python`` method) in ``flags_for_local_caster<T>()`` before
passing them to ``inner_caster.from_python()``. This allows nanobind
to prevent some casts that would produce dangling pointers or references.
* If ``inner_caster.from_python()`` succeeds, then also verify
``inner_caster.template can_cast<T>()`` before you execute
``inner_caster.operator cast_t<T>()``. A failure of
``can_cast()`` should be treated the same as a failure of
``from_python()``. This avoids the possibility of an exception
being raised through the noexcept ``load_python()`` method,
which would crash the interpreter.
The previous ``cast_flags::none_disallowed`` flag has been removed;
it existed to avoid one particular source of exceptions from a cast
operator, but ``can_cast<T>()`` now handles that problem more generally.
* ABI version 14.
.. rubric:: Footnote
.. [f1] The author of this library had somewhat of a revelation after
switching to a `new editor <https://neovim.io>`__ and experiencing the
benefits of interactive Python code completion and type checking for the
first time. This experience also showed how nanobind-based extension were
previously a second-class citizen in this typed world, prompting the changes
in this release.