Pyinstaller

------------------

Bugfix
~~~~~~

* (POSIX) Fix ``PyInstaller.depend.bindepend.resolve_library_path`` for
cases when ``ldconfig`` cache is not available (e.g., ``musl libc`` on
Alpine Linux). In such cases, the search code now distinguishes between
the case when fully suffixed library name is given (i.e., search for
exact match) and the case when library name has no suffix (i.e., search
for library with matching basename). (:issue:`8422`)
* (Windows) Fix mangling of path to the entry-point script when the script
is in the current working directory, and the path to this directory
contains two or more consecutive ``$`` or ``%`` characters. (:issue:`8434`)


Incompatible Changes
~~~~~~~~~~~~~~~~~~~~

* PyInstaller does not attempt to expand environment variables in paths
given via :option:`--workpath`, :option:`--distpath`, :option:`--specpath`,
and :option:`--additional-hooks-dir` anymore (note that other paths were
never subject to environment variable expansion in the first place).
Expansion of the starting tilde (``~``) into user's home directory is
still performed, as a work-around for tilde not being expanded by the
shell when passing arguments as ``--workpath=~/path/abc`` instead of
``--workpath ~/path/abc``. (:issue:`8441`)


Hooks
~~~~~

* Have ``sqlalchemy`` hook collect all dialects and plugins that are
registered via ``sqlalchemy.dialects`` and ``sqlalchemy.plugins``
entry-points. This ensures collection of 3rd party dialects and plugins
that may be available in the build environment (e.g., ``ibm-db-sa``).
(:issue:`8465`)
* The ``pywin32-ctypes`` hook now always collects the
``win32ctypes.core.ctypes``
modules, so that the ``ctypes`` backend is always available (i.e., even
if we also collect the ``cffi`` backend due to availability of ``cffi``
in the build environment). This fixes issues when ``cffi`` ends up
unavailable at run-time in spite of being available in the build environment
at build time (for example, due to explicit exclusion via
:option:`--exclude-module`
option). (:issue:`8544`)
* Update ``pkg_resources`` hook for compatibility with ``setuptools`` v70.0.0
and later (fix ``ModuleNotFoundError: No module named
'pkg_resources.extern'``). (:issue:`8554`)

------------------

Features
~~~~~~~~

* (Windows) Implement support for resolving executable's true location
when launched via a symbolic link. (:issue:`8300`)
* Implement an option to explicitly specify the bytecode optimization level
for collected python code, independent of the optimization level in the
python process under which PyInstaller is running. At the .spec file level,
this is controlled by optional ``optimize`` argument in the ``Analysis``
constructor. At the CLI level, this is controlled by new
:option:`--optimize` command-line option, which sets the ``optimize``
argument for ``Analysis`` as well as :ref:`interpreter run-time options
<specifying python interpreter options>` in the generated spec file.
See :ref:`bytecode optimization level` for details. (:issue:`8252`)


Bugfix
~~~~~~

* (macOS) Explicitly convert the value of ``version`` argument to ``BUNDLE``
into a string, in order to mitigate cases when user accidentally enters
an integer or a float. The version value ends up being written to
``Info.plist`` as the ``CFBundleShortVersionString`` entry, and if this
entry is not of a string type (for example, is an integer), the
generated .app bundle crashes at start. (:issue:`4466`)
* (Windows) Avoid trying to import ``PySimpleGUI`` in the subprocess that
analyzes dynamic library search modifications made by packages prior to
the binary dependency analysis. When imported for the first time,
``PySimpleGUI`` 5.x displays a "first-run" dialog, which poses a problem
for unattended PyInstaller builds running in a clean environment, for
example, in a CI pipeline. (:issue:`8396`)
* (Windows) Implement a work-around for running PyInstaller under python
process with ``-OO`` (or ``PYTHONOPTIMIZE=2``) with ``cffi`` installed.
We now temporarily disable import of ``cffi`` while importing
``pywin32-ctypes`` in ``PyInstaller.compat`` to ensure that ``ctypes``
backend is always used, as the ``cffi`` backend uses ``pycparser`` and
requires docstrings, which makes it incompatible with the ``-OO`` mode.
(:issue:`6345`)


Hooks
~~~~~

* Update ``PySide6.Qt3DRender`` hook for compatibility with ``PySide6``
6.7.0 (add hidden import for ``PySide6.QtOpenGL`` module). (:issue:`8404`)
* Update ``scipy.special._ufuncs`` hook for compatibility with SciPy 1.13.0
(add ``scipy.special._cdflib`` to hidden imports). (:issue:`8394`)


Bootloader
~~~~~~~~~~

* (Windows) Attempt to shorten the duration of spinning-wheel cursor when
launching applications built in ``windowed`` / ``noconsole`` mode.
(:issue:`8359`)


Documentation
~~~~~~~~~~~~~

* Add a new documentation section, :ref:`bytecode optimization level`,
which the describes the new canonical way to control bytecode
optimization level of the collected python code. (:issue:`8252`)
* Add a note to :ref:`specifying python interpreter options` to inform
user that setting the optimization level to the application's embedded
python interpreter by itself does not result in bytecode optimization of
modules that have been collected in byte-compiled form (i.e., the majority
of them). (:issue:`8252`)

------------------

Features
~~~~~~~~

* (Linux) Extend the mechanism for collection of ``.hmac`` files from
:issue:`8288` to ``.chk`` files that are used by NSS libraries.
(:issue:`8315`)


Bugfix
~~~~~~

* (Linux) Fix collection of ``QtWebEngineProcess`` helper when collecting
Qt (and ``PySide``/``PyQt`` bindings) installed via Linux distribution
packages. In such scenarios, we now force collection of the helper
executable into ``libexec`` directory inside the Qt sub-directory of
the bindings' package directory, in order to match the PyPI wheel layout.
(:issue:`8315`)
* (Linux) Fix regression that caused :func:`locale.getlocale` in
frozen applications created with PyInstaller v6.x to return ``(None, None)``
instead of user-preferred locale. (:issue:`8306`)
* (Windows) Avoid trying to import ``pyqtgraph.canvas`` in the subprocess
that analyzes dynamic library search modifications made by packages prior
to the binary dependency analysis. Trying to import ``pyqtgraph.canvas``
causes python interpreter to crash under certain circumstances (the
issue is present in ``pyqtgraph`` <= 0.13.3). (:issue:`8322`)
* (Windows) Fix collection of ``QtWebEngineProcess`` helper when
collecting ``PySide2`` and Qt installed via Anaconda on Windows.
The helper executable is now collected into top-level ``PySide2``
package directory, in order to match the PyPI wheel layout. (:issue:`8315`)
* (Windows) Suppress warnings about unresolvable UCRT DLLs
(``api-ms-win-*.dll``) on Windows 11. (:issue:`8339`)
* Fix bootloaders not being found when running an Intel build of Python on
Windows ARM64. (:issue:`8219`)


Incompatible Changes
~~~~~~~~~~~~~~~~~~~~

* PyInstaller now explicitly disallows attempts to collect multiple Qt
bindings packages (``PySide2``, ``PySide6``, ``PyQt5``, ``PyQt6``) into
a frozen application. When hooks for more than one top-level Qt bindings
package are executed, the build process is aborted with error message.
This restriction applies across all instances of ``Analysis`` within
a single build (i.e., a single .spec file).

If you encounter build errors caused by this new restriction, either
clean up your build environment (remove the bindings that you are not
using), or explicitly exclude the extraneous bindings using
:option:`--exclude-module`
(or equivalent ``excludes`` list passed as argument to ``Analysis`` in
the .spec file).

The automatic exclusion of extraneous bindings needs to be done via hooks on
per-package basis, so please `report problematic packages
<https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues>`_ so that we
can write hooks for them. (:issue:`8329`)


Hooks
~~~~~

* (Linux) When searching for dynamically-loaded NSS libraries during
collection of ``QtWebEnginge``, account for the possibility of said
libraries being either in a separate ``nss`` directory or in the main
library directory. This fixes problems with missing NSS libraries on
contemporary Linux distributions that do not use separate ``nss``
directory (anymore). (:issue:`8315`)
* Add a hook for ``pandas.io.clipboard`` to exclude the conditional
import of ``PyQt5`` from this module; the module primarily uses ``qtpy``
as its Qt bindings abstraction, and the conditional import of ``PyQt5``
interferes with Qt bindings selection done by our ``qtpy`` hook.
(:issue:`8329`)
* Add hook for ``qtpy`` to prevent collection of multiple available Qt
bindings. The hook attempts to select a single Qt bindings package
and exclude all other Qt bindings packages with the help of the
``PyInstaller.utils.hooks.qt.exclude_extraneous_qt_bindings``
helper. (:issue:`8329`)
* Extend hooks for ``matplotlib`` to prevent collection of multiple
available Qt bindings. The new hook for ``matplotlib.backends.qt_compat``
attempts to select a single Qt bindings package via the following
logic implemented in the
``PyInstaller.utils.hooks.qt.exclude_extraneous_qt_bindings``
helper: first, we check if hooks for any Qt bindings package have already
been run; if they had, those bindings are selected. If not, we check for
user-specified bindings in the ``QT_API`` environment variable; if valid
bindings name is specified, those bindings are selected. Otherwise, we
select one of available bindings. Once a Qt bindings package is selected,
the imports of all other Qt bindings packages are excluded from the
hooked package. (:issue:`8329`)
* Have run-time hooks for Qt bindings (``PySide2``, ``PySide6``, ``PyQt5``,
and ``PyQt6``) check for presence of the embedded ``:/qt/etc/qt.conf``
resource, and if not present, inject their own version. This
aims to ensure that the bundled Qt is always relocatable, even if the
package does not perform injection of embedded ``qt.conf`` file (most
notably, this seems to be the case with ``PySide2`` collected from
Linux distribution packages, and ``PySide2`` collected from Anaconda
on Windows, Linux, and macOS). (:issue:`8315`)
* PyInstaller now explicitly disallows attempts to collect multiple Qt
bindings packages (``PySide2``, ``PySide6``, ``PyQt5``, ``PyQt6``) into
a frozen application. When hooks for more than one top-level Qt bindings
package are executed, the build process is aborted with error message
that informs user of the situation and what to do about it (i.e., exclusion
of extraneous packages). The limitation applies to all analyses within a
spec file. (:issue:`8329`)
* Remove run-time hook for ``win32com``, as per discussion in issue:`8309`.
(:issue:`8313`)
* Update hook for ``matplotlib.backends`` to include ``QtAgg`` and ``Gtk4Agg``
in the list of backend candidates. (:issue:`8334`)


Bootloader
~~~~~~~~~~

* Have bootloader set the ``configure_locale`` field in the interpreter
pre-config structure, so that user-preferred locale is set during
interpreter pre-initialization. (:issue:`8306`)


Bootloader build
~~~~~~~~~~~~~~~~

* The target architecture on Windows using MSVC now defaults to that of the
current Python environment – not the current OS. (:issue:`8219`)

------------------

Features
~~~~~~~~

* (Linux) Collect ``.hmac`` files accompanying shared libraries, if such files
are available. This allows frozen application to run on FIPS-enabled Red Hat
Enterprise systems, where HMAC is required by self-check implemented by the
OpenSSL crypto library. Furthermore, ensure that shared libraries with
accompanying ``.hmac`` files are exempted from any additional processing (for
example, when building with :option:`--strip` option) to avoid invalidating
the HMAC. (:issue:`8273`)
* (Windows) Make bootloader codepaths involved in creation of temporary
directories for ``onefile`` builds AppContainer-aware. If the process runs
inside an AppContainer, the temporary directory's DACL needs to explicitly
include the AppContainerSID, otherwise the directory becomes inaccessible to
the process. (:issue:`8291`)
* (Windows) Make Windows implementation of PyInstaller's
``_pyi_rth_utils.tempdir.secure_mkdir`` (used by ``matplotlib`` and
``win32com`` run-time hooks to create temporary directories)
AppContainer-aware. If the process runs inside an AppContainer, the temporary
directory's DACL needs to explicitly include the AppContainerSID, otherwise
the directory becomes inaccessible to the process. (:issue:`8290`)
* Implement strict Qt dependency validation for collection of Qt plugins and QML
components/plugins. We now perform preliminary binary dependency analysis of
the plugins, and automatically exclude plugins that have at least one missing
Qt dependency. This prevents collection of plugins that cannot be used anyway
because of a missing Qt shared library (that is, for example, omitted from a
PyPI wheel). Furthermore, we disallow Qt dependencies of a plugin to be
resolved outside of the primary location of Qt shared libraries, in order to
prevent missing dependencies from pulling in Qt libraries from alternative
locations that happen to be in the search path (for example, when using
``PyQt5`` PyPI wheels while also having a system-installed Qt5 on Linux, a
Homebrew-installed Qt5 on macOS, or a custom Windows Qt5 build that happens to
be in ``PATH``). (:issue:`8226`)


Bugfix
~~~~~~

* (Linux) Prevent collection of ``libcuda.so.1``, which is part of NVIDIA
driver and must match the rest of the driver's components. Collecting
a copy might lead to issues when build and target system use different
versions of NVIDIA driver. (:issue:`8278`)
* (macOS) When validating the macOS SDK version of collected binaries,
handle errors raised by ``osxutils.get_macos_sdk_version``; log a
warning about failed version query, and add the offending binary to
the list of potentially problematic binaries to warn the user about.
(:issue:`8220`)
* Fix ``pkgutil.iter_modules`` override to gracefully handle cases when
the given path corresponds to a module instead of a package. (:issue:`8191`)
* Prevent Qt and QML plugins with missing Qt dependencies in the
``PySide2``, ``PyQt5``, ``PySide6``, and ``PyQt6`` PyPI wheels from
pulling in Qt shared libraries from alternative locations (for example,
system-installed Qt on Linux, Homebrew-installed Qt on macOS, or
a custom Windows Qt build that happens to be in ``PATH``), and resulting
in a frozen application that contains an incompatible mix of Qt libraries.
(:issue:`8087`)
* Switch the hashing function in PyInstaller's binary cache from MD5 to
SHA1, as the former cannot be used on FIPS-enabled Red Hat Enterprise
Linux systems. (:issue:`8288`)
* When trying to run ``pyinstaller`` (or equivalent ``python -m PyInstaller``)
against non-existing script file(s), exit immediately - without trying
to write the .spec file and building it. This prevents us from overwriting
an existing (and customized) .spec file if user makes a typo in the .spec
file's suffix when trying to build it, for example, ``pyinstaller
program.spec``. (:issue:`8279`)


Hooks
~~~~~

* (macOS) Have ``PySide6`` and ``PyQt6`` run-time hooks prepend
``sys._MEIPASS`` to ``DYLD_LIBRARY_PATH`` in POSIX builds, in order
to ensure that ``QtNetwork`` discovers the bundled copy of the OpenSSL
shared library. (:issue:`8226`)
* Extend the OpenSSL shared library collection in the ``QtNetwork`` hook
helper for ``PySide2``, ``PyQt5``, ``PySide6``, and ``PyQt6`` to
cover all applicable versions of OpenSSL (1.0.2, 1.1.x, 3.x). In
addition to Windows, the OpenSSL shared library is now also collected
on Linux and macOS. (:issue:`8226`)


Bootloader
~~~~~~~~~~

* (Windows) Update the bundled zlib sources to v1.3.1. (:issue:`8292`)


Documentation
~~~~~~~~~~~~~

* Add a new documentation chapter, called :ref:`common issues`, to cover
topics such as launching external programs from frozen applications,
multi-processing via :mod:`multiprocessing` (specifically, the requirement
to call :func:`multiprocessing.freeze_support`), use of symbolic links in
POSIX builds in PyInstaller >= 6.0 and its implications for distribution
(e.g., when copying frozen application, or creating ``zip`` archives),
:data:`sys.stdout` and :data:`sys.stderr` being :data:`None` in Windows
no-console builds. (:issue:`8214`)
* Cleanup docstrings to remove mention of ``exec_command_stdout``.
(:issue:`8173`)
* Update the :ref:`macOS app bundles` section to reflect the layout of
macOS app bundles as produced by PyInstaller 6.0 and later. Add a note
to discourage use of onefile .app bundles. (:issue:`8214`)
* Update the introduction part of the :ref:`understanding pyinstaller hooks`
section. (:issue:`8214`)

------------------

Bugfix
~~~~~~

* (Linux) Optimize the automatic binary-vs-data classification by avoiding
``objdump`` based check on files that do not have ELF signature. This
mitigates noticeably longer analysis times for projects with large number of
(data) files. (:issue:`8148`)
* (Windows) Add Windows error code 110 (``ERROR_OPEN_FAILED``) to the list of
error codes eligible for the retry mechanism that attempts to mitigate build
failures due to anti-virus program interference. (:issue:`8138`)
* (Windows) Fix issue with non-functional :func:`time.sleep()` when building
program with Python <= 3.8.6 or Python 3.9.0. (:issue:`8104`)
* (Windows) Fix issue with splash screen in ``onefile`` mode failing to extract
``VCRUNTIME140.dll`` from the archive due to character-case mismatch. We now
perform case-insensitive comparison between the name listed in splash
dependency list and the names in archive TOC. (:issue:`8103`)
* Fix PEP 597 EncodingWarnings when :envvar:`PYTHONWARNDEFAULTENCODING` is set
to true. (:issue:`8117`)
* Fix pre-safe-import hooks for ``six.moves``, ``urllib3.packages.six.moves``,
and ``setuptools.extern.six.moves`` to gracefully handle cases when the
corresponding ``six`` package is unavailable, as the hook may end up being
executed even in that case. (:issue:`8145`)
* Fix symbolic link tracking in ``MERGE`` processing, so that distinct symbolic
links with same relative target (e.g. ``Current -> A`` symbolic links in Qt
.framework bundles collected on macOS) are properly processed, and kept in the
original TOC upon their first occurrence. (:issue:`8124`)


Hooks
~~~~~

* Add hook for ``gi.repository.DBus``. (:issue:`8149`)
* Add hooks for ``gi.repository.AppIndicator3`` and
``gi.repository.AyatanaAppIndicator3``. (:issue:`8149`)


Bootloader
~~~~~~~~~~

* When setting up embedded Python interpreter configuration, set
``PyConfig.install_signal_handlers=1`` to install signal handlers.
This matches the behavior of PyInstaller 5.x bootloaders, where interpreter
was initialized via ``Py_Initialize()``, which in turn calls
``Py_InitializeEx(1)``, i.e., with ``install_sigs=1``. (:issue:`8105`)

------------------

Features
~~~~~~~~

* (macOS) At the end of analysis, verify the macOS SDK version reported
by binaries to be collected, and warn when the version is either invalid
(0.0.0) or too low (< 10.9.0). Such binaries will likely cause issues
with code-signing and hardened runtime. (:issue:`8043`)
* If the ``argcomplete`` Python module is installed, PyInstaller will use it
enable tab completion for its CLI tools. PyInstaller CLIs can still be used
without this optional dependency. To install ``argcomplete`` with
PyInstaller, you can put ``pyinstaller[completion]`` in your dependencies.
See also `the argcomplete documentation
<https://kislyuk.github.io/argcomplete/>`_. (:issue:`8008`)


Bugfix
~~~~~~

* (macOS) Fix the bug in binary processing and caching that would update
the binary cache index before performing macOS-specific processing
(architecture validation, path rewriting). If, for example, architecture
validation failed during a build, subsequent build attempts with
enabled binary cache (i.e., without the :option:`--clean` option) would
pick up the partially-processed binary file from the cache, bypassing the
architecture validation. NOTE: the existing binary caches need to be
purged manually (using :option:`--clean` option once) for the fix to take
effect! (:issue:`8068`)
* (macOS) Prevent collection of ``.DS_Store`` files, which might be present
in build environment's package directories after user navigated them using
the Finder app. (:issue:`8042`)
* (Windows) Fix marshal error at the start of binary dependency analysis,
caused by inferred DLL search path ending up an instance of
:class:`pathlib.Path` instead of :class:`str`. (:issue:`8081`)
* Bump the required ``packaging`` version to 22.0, which is required for
proper handling of metadata that contains markers with ``extra``\ s.
(:issue:`8061`)
* Fix erroneous DLL parent path preservation when :data:`sys.base_prefix`
itself is a symbolic link. In such case, we need to exclude both
resolved and unresolved path variant for ``sys.base_prefix``, in order to
prevent either from ending up in the list of directories for which DLL
parent paths are preserved. Failing to do so, for example, caused
``_ctypes`` failing to load in an application build on Windows with
Python installed via ``scoop``, due to ``libffi-8.dll`` having spuriously
preserved the parent directory path instead of being collected to top-level
application directory. (:issue:`8023`)
* Fix matching of pre-release versions in
:func:`PyInstaller.utils.hooks.check_requirement` and
:func:`PyInstaller.utils.hooks.is_module_satisfies`. Both functions now
match pre-release versions, which restores the behavior of the old
``pkg_resources``-based implementation from PyInstaller < 6.0
that is implicitly expected by existing hooks. (:issue:`8093`)
* If the entry-point script has no suffix, append the ``.py`` suffix
to the filename passed to the ``compile`` function when byte-compiling
the script for collection. This ensures that the entry-point script
filename never coincides with executable filename, especially in POSIX
builds, where executables have no suffix either (and their name is based
on the entry-point script basename by default). Entry-point script having
the same filename as the executable causes issues when ``traceback``
(and ``linecache``) try to access source code for it, an in the process
end up reading the executable file if it happens to be in the current
working directory. (:issue:`8046`)
* Improve speed of :func:`pkgutil.iter_modules` override, especially in cases
when the function is called multiple times. (:issue:`8058`)
* Load PyInstaller hooks using :pep:`451` ``importlib.abc.Loader.exec_module``
instead of deprecated :pep:`302` ``importlib.abc.Loader.load_module``.
(:issue:`8031`)
* Prevent an attempt at relative import of a missing (optional) sub-module
within a package (e.g., ``from .module import something``) from tricking
the modulegraph/analysis into collecting an unrelated but eponymous
top-level module. (:issue:`8010`)


Hooks
~~~~~

* Add hook for ``PySide6.QtGraphs`` that was introduced in ``PySide6`` 6.6.0.
(:issue:`8021`)
* Add hooks for ``distutils.command.check`` and
``setuptools._distutils.command.check`` that prevent unnecessary
collection of ``docutils`` (which in turn triggers collection of
``pygments``, ``PIL``, etc.). (:issue:`8053`)
* Deduplicate and sort the list of discovered/selected ``matplotlib``
backends before displaying it in log messages, to avoid giving
impression that they are collected multiple times. (:issue:`8009`)
* Update ``PySide6`` hooks for compatibility with ``PySide6`` 6.6.0 and
python 3.12. (:issue:`8021`)