Pybind11

Changes:

- If possible, attach Python exception with `py::raise_from` to `TypeError` when casting from C++ to Python. This will give additional info if Python exceptions occur in the caster. Adds a test case of trying to convert a set from C++ to Python when the hash function is not defined in Python. [\3605](https://github.com/pybind/pybind11/pull/3605)
- Add a mapping of C++11 nested exceptions to their Python exception equivalent using `py::raise_from`. This attaches the nested exceptions in Python using the `__cause__` field. [\3608](https://github.com/pybind/pybind11/pull/3608)
- Propagate Python exception traceback using `raise_from` if a pybind11 function runs out of overloads. [\3671](https://github.com/pybind/pybind11/pull/3671)
- `py::multiple_inheritance` is now only needed when C++ bases are hidden from pybind11. [\3650](https://github.com/pybind/pybind11/pull/3650) and [\#3659](https://github.com/pybind/pybind11/pull/3659)

Bug fixes:

- Remove a boolean cast in `numpy.h` that causes MSVC C4800 warnings when compiling against Python 3.10 or newer. [\3669](https://github.com/pybind/pybind11/pull/3669)
- Render `py::bool_` and `py::float_` as `bool` and `float` respectively. [\3622](https://github.com/pybind/pybind11/pull/3622)

Build system improvements:

- Fix CMake extension suffix computation on Python 3.10+. [\3663](https://github.com/pybind/pybind11/pull/3663)
- Allow `CMAKE_ARGS` to override CMake args in pybind11's own `setup.py`. [\3577](https://github.com/pybind/pybind11/pull/3577)
- Remove a few deprecated c-headers. [\3610](https://github.com/pybind/pybind11/pull/3610)
- More uniform handling of test targets. [\3590](https://github.com/pybind/pybind11/pull/3590)
- Add clang-tidy readability check to catch potentially swapped function args. [\3611](https://github.com/pybind/pybind11/pull/3611)

This is the last version to support Python 2.7 and 3.5.

New Features:

- Allow `py::args` to be followed by other arguments; the remaining arguments are implicitly keyword-only, as if a `py::kw_only{}` annotation had been used. [3402](https://github.com/pybind/pybind11/pull/3402)

Changes:

- Make str/bytes/memoryview more interoperable with `std::string_view`. [3521](https://github.com/pybind/pybind11/pull/3521)
- Replace `_` with `const_name` in internals, avoid defining `pybind::_` if `_` defined as macro (common gettext usage) [3423](https://github.com/pybind/pybind11/pull/3423)

Bug fixes:

- Fix a rare warning about extra copy in an Eigen constructor. [3486](https://github.com/pybind/pybind11/pull/3486)
- Fix caching of the C++ overrides. [3465](https://github.com/pybind/pybind11/pull/3465)
- Add missing `std::forward` calls to some `cpp_function` overloads. [3443](https://github.com/pybind/pybind11/pull/3443)
- Support PyPy 7.3.7 and the PyPy3.8 beta. Test python-3.11 on PRs with the `python dev` label. [3419](https://github.com/pybind/pybind11/pull/3419)
- Replace usage of deprecated `Eigen::MappedSparseMatrix` with `Eigen::Map<Eigen::SparseMatrix<...>>` for Eigen 3.3+. [3499](https://github.com/pybind/pybind11/pull/3499)
- Tweaks to support Microsoft Visual Studio 2022. [3497](https://github.com/pybind/pybind11/pull/3497)

Build system improvements:

- Nicer CMake printout and IDE organisation for pybind11's own tests. [3479](https://github.com/pybind/pybind11/pull/3479)
- CMake: report version type as part of the version string to avoid a spurious space in the package status message. [3472](https://github.com/pybind/pybind11/pull/3472)
- Flags starting with `-g` in `$CFLAGS` and `$CPPFLAGS` are no longer overridden by `.Pybind11Extension`. [3436](https://github.com/pybind/pybind11/pull/3436)
- Ensure ThreadPool is closed in `setup_helpers`. [3548](https://github.com/pybind/pybind11/pull/3548)
- Avoid LTS on `mips64` and `ppc64le` (reported broken). [3557](https://github.com/pybind/pybind11/pull/3557)

Changes and additions:
- The simple namespace creation shortcut added in 2.8.0 was deprecated due to usage of CPython internal API, and will be removed soon. Use `py::module_::import("types").attr("SimpleNamespace")`. [\3374](https://github.com/pybinyyd/pybind11/pull/3374)
- Add C++ Exception type to throw and catch `AttributeError`. Useful for defining custom `__setattr__` and `__getattr__` methods. [\3387](https://github.com/pybind/pybind11/pull/3387)

Fixes:
- Fixed the potential for dangling references when using properties with `std::optional` types. [\3376](https://github.com/pybind/pybind11/pull/3376)
- Modernize usage of `PyCodeObject` on Python 3.9+ (moving toward support for Python 3.11a1) [\3368](https://github.com/pybind/pybind11/pull/3368)
- A long-standing bug in `eigen.h` was fixed (originally PR \3343). The bug was unmasked by newly added `static_assert`'s in the Eigen 3.4.0 release. [\3352](https://github.com/pybind/pybind11/pull/3352)
- Support multiple raw inclusion of CMake helper files (Conan.io does this for multi-config generators). [\3420](https://github.com/pybind/pybind11/pull/3420)
- Fix harmless warning on upcoming CMake 3.22. [\3368](https://github.com/pybind/pybind11/pull/3368)
- Fix 2.8.0 regression with MSVC 2017 + C++17 mode + Python 3. [\3407](https://github.com/pybind/pybind11/pull/3407)
- Fix 2.8.0 regression that caused undefined behavior (typically segfaults) in `make_key_iterator`/`make_value_iterator` if dereferencing the iterator returned a temporary value instead of a reference. [\3348](https://github.com/pybind/pybind11/pull/3348)

New features:

- Added `py::raise_from` to enable chaining exceptions. [3215](https://github.com/pybind/pybind11/pull/3215)
- Allow exception translators to be optionally registered local to a module instead of applying globally across all pybind11 modules. Use `register_local_exception_translator(ExceptionTranslator&& translator)` instead of `register_exception_translator(ExceptionTranslator&& translator)` to keep your exception remapping code local to the module. [2650](https://github.com/pybind/pybind11/pull/2650)
- Add `make_simple_namespace` function for instantiating Python `SimpleNamespace` objects. [2840](https://github.com/pybind/pybind11/pull/2840)
- `pybind11::scoped_interpreter` and `initialize_interpreter` have new arguments to allow `sys.argv` initialization. [2341](https://github.com/pybind/pybind11/pull/2341)
- Allow Python builtins to be used as callbacks in CPython. [1413](https://github.com/pybind/pybind11/pull/1413)
- Added `view` to view arrays with a different datatype. [987](https://github.com/pybind/pybind11/pull/987)
- Implemented `reshape` on arrays. [984](https://github.com/pybind/pybind11/pull/984)
- Enable defining custom `__new__` methods on classes by fixing bug preventing overriding methods if they have non-pybind11 siblings. [3265](https://github.com/pybind/pybind11/pull/3265)
- Add `make_value_iterator()`, and fix `make_key_iterator()` to return references instead of copies. [3293](https://github.com/pybind/pybind11/pull/3293)
- Improve the classes generated by `bind_map`: [3310](https://github.com/pybind/pybind11/pull/3310)
- Change `.items` from an iterator to a dictionary view.
- Add `.keys` and `.values` (both dictionary views).
- Allow `__contains__` to take any object.
- `pybind11::custom_type_setup` was added, for customizing the `PyHeapTypeObject` corresponding to a class, which may be useful for enabling garbage collection support, among other things. [3287](https://github.com/pybind/pybind11/pull/3287)

Changes:

- Set `__file__` constant when running `eval_file` in an embedded interpreter. [3233](https://github.com/pybind/pybind11/pull/3233)
- Python objects and (C++17) `std::optional` now accepted in `py::slice` constructor. [1101](https://github.com/pybind/pybind11/pull/1101)
- The pybind11 proxy types `str`, `bytes`, `bytearray`, `tuple`, `list` now consistently support passing `ssize_t` values for sizes and indexes. Previously, only `size_t` was accepted in several interfaces. [3219](https://github.com/pybind/pybind11/pull/3219)
- Avoid evaluating `PYBIND11_TLS_REPLACE_VALUE` arguments more than once. [3290](https://github.com/pybind/pybind11/pull/3290)

Fixes:

- Bug fix: enum value's `__int__` returning non-int when underlying type is bool or of char type [1334](https://github.com/pybind/pybind11/pull/1334)
- Fixes bug in setting error state in Capsule's pointer methods. [3261](https://github.com/pybind/pybind11/pull/3261)
- A long-standing memory leak in `py::cpp_function::initialize` was fixed. [3229](https://github.com/pybind/pybind11/pull/3229)
- Fixes thread safety for some `pybind11::type_caster` which require lifetime extension, such as for `std::string_view`. [3237](https://github.com/pybind/pybind11/pull/3237)
- Restore compatibility with gcc 4.8.4 as distributed by ubuntu-trusty, linuxmint-17. [3270](https://github.com/pybind/pybind11/pull/3270)

Build system improvements:

- Fix regression in CMake Python package config: improper use of absolute path. [3144](https://github.com/pybind/pybind11/pull/3144)
- Cached Python version information could become stale when CMake was re-run with a different Python version. The build system now detects this and updates this information. [3299](https://github.com/pybind/pybind11/pull/3299)
- Specified UTF8-encoding in setup.py calls of open(). [3137](https://github.com/pybind/pybind11/pull/3137)
- Fix a harmless warning from CMake 3.21 with the classic Python discovery. [3220](https://github.com/pybind/pybind11/pull/3220)
- Eigen repo and version can now be specified as cmake options. [3324](https://github.com/pybind/pybind11/pull/3324)

Backend and tidying up:

- Reduced thread-local storage required for keeping alive temporary data for type conversion to one key per ABI version, rather than one key per extension module. This makes the total thread-local storage required by pybind11 2 keys per ABI version. [3275](https://github.com/pybind/pybind11/pull/3275)
- Optimize NumPy array construction with additional moves. [3183](https://github.com/pybind/pybind11/pull/3183)
- Conversion to `std::string` and `std::string_view` now avoids making an extra copy of the data on Python >= 3.3. [3257](https://github.com/pybind/pybind11/pull/3257)
- Remove const modifier from certain C++ methods on Python collections (`list`, `set`, `dict`) such as (`clear()`, `append()`, `insert()`, etc...) and annotated them with `py-non-const`.
- Enable readability `clang-tidy-const-return` and remove useless consts. [3254](https://github.com/pybind/pybind11/pull/3254) [#3194](https://github.com/pybind/pybind11/pull/3194)
- The clang-tidy `google-explicit-constructor` option was enabled. [3250](https://github.com/pybind/pybind11/pull/3250)
- Mark a pytype move constructor as noexcept (perf). [3236](https://github.com/pybind/pybind11/pull/3236)
- Enable clang-tidy check to guard against inheritance slicing. [3210](https://github.com/pybind/pybind11/pull/3210)
- Legacy warning suppression pragma were removed from eigen.h. On Unix platforms, please use -isystem for Eigen include directories, to suppress compiler warnings originating from Eigen headers. Note that CMake does this by default. No adjustments are needed for Windows. [3198](https://github.com/pybind/pybind11/pull/3198)
- Format pybind11 with isort consistent ordering of imports [3195](https://github.com/pybind/pybind11/pull/3195)
- The warnings-suppression "pragma clamp" at the top/bottom of pybind11 was removed, clearing the path to refactoring and IWYU cleanup. [3186](https://github.com/pybind/pybind11/pull/3186)
- Enable most bugprone checks in clang-tidy and fix the found potential bugs and poor coding styles. [3166](https://github.com/pybind/pybind11/pull/3166)
- Add `clang-tidy-readability` rules to make boolean casts explicit improving code readability. Also enabled other misc and readability clang-tidy checks. [3148](https://github.com/pybind/pybind11/pull/3148)
- Move object in `.pop()` for list. [3116](https://github.com/pybind/pybind11/pull/3116)

Minor missing functionality added:

- Allow Python builtins to be used as callbacks in CPython. [1413](https://github.com/pybind/pybind11/pull/1413)

Bug fixes:

- Fix regression in CMake Python package config: improper use of absolute path. [3144](https://github.com/pybind/pybind11/pull/3144)
- Fix Mingw64 and add to the CI testing matrix. [3132](https://github.com/pybind/pybind11/pull/3132)
- Specified UTF8-encoding in setup.py calls of open(). [3137](https://github.com/pybind/pybind11/pull/3137)
- Add clang-tidy-readability rules to make boolean casts explicit improving code readability. Also enabled other misc and readability clang-tidy checks. [3148](https://github.com/pybind/pybind11/pull/3148)
- Move object in `.pop()` for list. [3116](https://github.com/pybind/pybind11/pull/3116)

Backend and tidying up:

- Removed and fixed warning suppressions. [3127](https://github.com/pybind/pybind11/pull/3127) [#3129](https://github.com/pybind/pybind11/pull/3129) [#3135](https://github.com/pybind/pybind11/pull/3135) [#3141](https://github.com/pybind/pybind11/pull/3141) [#3142](https://github.com/pybind/pybind11/pull/3142) [#3150](https://github.com/pybind/pybind11/pull/3150) [#3152](https://github.com/pybind/pybind11/pull/3152) [#3160](https://github.com/pybind/pybind11/pull/3160) [#3161](https://github.com/pybind/pybind11/pull/3161)

New features:

- Enable `py::implicitly_convertible<py::none, ...>` for `py::class_`-wrapped types. [3059](https://github.com/pybind/pybind11/pull/3059)
- Allow function pointer extraction from overloaded functions. [2944](https://github.com/pybind/pybind11/pull/2944)
- NumPy: added `.char_()` to type which gives the NumPy public `char` result, which also distinguishes types by bit length (unlike `.kind()`). [2864](https://github.com/pybind/pybind11/pull/2864)
- Add `pybind11::bytearray` to manipulate `bytearray` similar to `bytes`. [2799](https://github.com/pybind/pybind11/pull/2799)
- `pybind11/stl/filesystem.h` registers a type caster that, on C++17/Python 3.6+, converts `std::filesystem::path` to `pathlib.Path` and any `os.PathLike` to `std::filesystem::path`. [2730](https://github.com/pybind/pybind11/pull/2730)
- A `PYBIND11_VERSION_HEX` define was added, similar to `PY_VERSION_HEX`. [3120](https://github.com/pybind/pybind11/pull/3120)

Changes:

- `py::str` changed to exclusively hold `PyUnicodeObject`. Previously `py::str` could
also hold `bytes`, which is probably surprising, was never documented, and can mask bugs (e.g. accidental use of `py::str` instead of `py::bytes`). [2409](https://github.com/pybind/pybind11/pull/2409)
- Add a safety guard to ensure that the Python GIL is held when C++ calls back into Python via `object_api<>::operator()` (e.g. `py::function` `__call__`). (This feature is available for Python 3.6+ only.) [2919](https://github.com/pybind/pybind11/pull/2919)
- Catch a missing `self` argument in calls to `__init__()`. [2914](https://github.com/pybind/pybind11/pull/2914)
- Use `std::string_view` if available to avoid a copy when passing an object to a `std::ostream`. [3042](https://github.com/pybind/pybind11/pull/3042)
- An important warning about thread safety was added to the `iostream.h` documentation; attempts to make `py::scoped_ostream_redirect` thread safe have been removed, as it was only partially effective. [2995](https://github.com/pybind/pybind11/pull/2995)

Fixes:

- Performance: avoid unnecessary strlen calls. [3058](https://github.com/pybind/pybind11/pull/3058)
- Fix auto-generated documentation string when using `const T` in `pyarray_t`. [3020](https://github.com/pybind/pybind11/pull/3020)
- Unify error messages thrown by `simple_collector`/`unpacking_collector`. [3013](https://github.com/pybind/pybind11/pull/3013)
- `pybind11::builtin_exception` is now explicitly exported, which means the types included/defined in different modules are identical, and exceptions raised in different modules can be caught correctly. The documentation was updated to explain that custom exceptions that are used across module boundaries need to be explicitly exported as well. [2999](https://github.com/pybind/pybind11/pull/2999)
- Fixed exception when printing UTF-8 to a `scoped_ostream_redirect`. [2982](https://github.com/pybind/pybind11/pull/2982)
- Pickle support enhancement: `setstate` implementation will attempt to `setattr` `__dict__` only if the unpickled `dict` object is not empty, to not force use of `py::dynamic_attr()` unnecessarily. [2972](https://github.com/pybind/pybind11/pull/2972)
- Allow negative timedelta values to roundtrip. [2870](https://github.com/pybind/pybind11/pull/2870)
- Fix unchecked errors could potentially swallow signals/other exceptions. [2863](https://github.com/pybind/pybind11/pull/2863)
- Add null pointer check with `std::localtime`. [2846](https://github.com/pybind/pybind11/pull/2846)
- Fix the `weakref` constructor from `py::object` to create a new `weakref` on conversion. [2832](https://github.com/pybind/pybind11/pull/2832)
- Avoid relying on exceptions in C++17 when getting a `shared_ptr` holder from a `shared_from_this` class. [2819](https://github.com/pybind/pybind11/pull/2819)
- Allow the codec's exception to be raised instead of `RuntimeError` when casting from `py::str` to `std::string`. [2903](https://github.com/pybind/pybind11/pull/2903)

Build system improvements:

- In `setup_helpers.py`, test for platforms that have some multiprocessing features but lack semaphores, which `ParallelCompile` requires. [3043](https://github.com/pybind/pybind11/pull/3043)
- Fix `pybind11_INCLUDE_DIR` in case `CMAKE_INSTALL_INCLUDEDIR` is absolute. [3005](https://github.com/pybind/pybind11/pull/3005)
- Fix bug not respecting `WITH_SOABI` or `WITHOUT_SOABI` to CMake. [2938](https://github.com/pybind/pybind11/pull/2938)
- Fix the default `Pybind11Extension` compilation flags with a Mingw64 python. [2921](https://github.com/pybind/pybind11/pull/2921)
- Clang on Windows: do not pass `/MP` (ignored flag). [2824](https://github.com/pybind/pybind11/pull/2824)
- `pybind11.setup_helpers.intree_extensions` can be used to generate `Pybind11Extension` instances from cpp files placed in the Python
package source tree. [2831](https://github.com/pybind/pybind11/pull/2831)

Backend and tidying up:

- Enable clang-tidy performance, readability, and modernization checks throughout the codebase to enforce best coding practices. [3046](https://github.com/pybind/pybind11/pull/3046), [#3049](https://github.com/pybind/pybind11/pull/3049), [#3051](https://github.com/pybind/pybind11/pull/3051), [#3052](https://github.com/pybind/pybind11/pull/3052), [#3080](https://github.com/pybind/pybind11/pull/3080), and [#3094](https://github.com/pybind/pybind11/pull/3094)
- Checks for common misspellings were added to the pre-commit hooks. [3076](https://github.com/pybind/pybind11/pull/3076)
- Changed `Werror` to stricter `Werror-all` for Intel compiler and fixed minor issues. [2948](https://github.com/pybind/pybind11/pull/2948)
- Fixed compilation with GCC \< 5 when the user defines `_GLIBCXX_USE_CXX11_ABI`. [2956](https://github.com/pybind/pybind11/pull/2956)
- Added nox support for easier local testing and linting of contributions. [3101](https://github.com/pybind/pybind11/pull/3101) and [#3121](https://github.com/pybind/pybind11/pull/3121)
- Avoid RTD style issue with docutils 0.17+. [3119](https://github.com/pybind/pybind11/pull/3119)
- Support pipx run, such as `pipx run pybind11 --include` for a quick compile. [3117](https://github.com/pybind/pybind11/pull/3117)