Scons

From Mats Wichmann
- Fix a problem in 4.5.0 where using something like the following code
will cause a Clone()'d environment to share the CPPDEFINES with the
original Environment() which was cloned. Causing leakage of changes
to CPPDEFINES when they should be completely independent after the Clone.
env=Environment(CPPDEFINES=['a'])
env.Append(CPPDEFINES=['b']) (or AppendUnique,Prepend,PrependUnique)
env1=env.Clone()
env1.Append(CPPDEFINES=['c']) (or any other modification, but not overwriting CPPDEFINES
Now env['CPPDEFINES'] will contain 'c' when it should not.

From Anatoli Babenia:
- Do not initialize DefaultEnvironment when calling EnsureSConsVersion(),
EnsurePythonVersion(), Exit(), GetLaunchDir() and SConscriptChdir().
- Remove unused private method SConsEnvironment._exceeds_version().

From William Deegan:
- Added ValidateOptions() which will check that all command line options are in either
those specified by SCons itself, or by AddOption() in SConstruct/SConscript. It should
not be called until all AddOption() calls are completed. Resolves Issue 4187
- Refactored SCons/Taskmaster into a package. Moved SCons/Jobs.py into that package.
NOTE: If you hook into SCons.Jobs, you'll have to change that to use SCons.Taskmaster.Jobs
- Changed the Taskmaster trace logic to use python's logging module. The output formatting
should remain (mostly) the same. Minor update to unittest for this to adjust for 1 less newline.
- Migrated logging logic for --taskmastertrace to use Python's logging module. Added logging
to NewParallel Job class (Andrew Morrow's new parallel job implementation)
- Ninja: Fix execution environment sanitation for launching ninja. Previously if you set an
execution environment variable set to a python list it would crash. Now it
will create a string joining the list with os.pathsep
- Move execution environment sanitation from Action._subproc() to
SCons.Util.sanitize_shell_env(ENV)
- Moved rpm and debian directories under packaging
- Added logic to help packagers enable reproducible builds into packaging/etc/. Please
read packaging/etc/README.txt if you are interested.
- Added --experimental=tm_v2, which enables Andrew Morrow's new NewParallel Job implementation.
This should scale much better for highly parallel builds. You can also enable this via SetOption().
- Fixed command line argument --diskcheck: previously a value of 'none' was ignored.
SetOption('diskcheck','none') is unaffected, as it did not have the problem.
- Added overrides argument to SCons.Subst.scons_subst(), subst_list(), subst(), and Action's process(),
strfunction(). This allows passing a dictionary of envvars to override when evaluating subst()'d strings/lists
- Fixed Issue 4275 - when outputting compilation db and TEMPFILE was in use, the compilation db would have
command lines using the generated tempfile for long command lines, instead of the full command line for
the compilation step for the source/target pair.
- Renamed the qt tools to qt3 since the logic in that tool is only for QT version 3. Renamed all env vars
which affect qt3 from QT_ to QT3_. If you are still using SCons to build QT 3 code, you'll need to update
your SConscripts. Note that using 'qt' tool has been deprecated for some time.

From David H:
- Added JAVAPROCESSORPATH construction variable which populates -processorpath.
- Updated JavaScanner to scan JAVAPROCESSORPATH.

From Nickolai Korshunov
- Added FILE_ENCODING, to allow explicitly setting the text encoding for files
written by the Textfile() and Substfile() builders. If not specified, Textfile() and Substfile() builders
will write files as UTF-8. Fixed Issue 4302.

From Dan Mezhiborsky:
- Add newline to end of compilation db (compile_commands.json).

From Daniel Moody:
- Added error message to handle the case when SCons attempts to retrieve all the targets
for a specified builder from the CacheDir, fails to do so, and then runs into an error
when deleting the files which were retrieved. Previously if this happened there was no
errors or warnings.
- Fix issue 2757, where Configure checks that perform a check which reads a modified source
(including program, source or header file(s)) would incorrectly mark that file "up to date" so the
actual build would not see the file as modified. Leading to incorrect incremental builds.
Now configure checks now clear node info for non conftest nodes, so they will be re-evaluated for
the real taskmaster run when the build commences.

From Andrew Morrow
- Avoid returning UniqueList for `children` and other `Executor` APIs. This type
iterates more slowly than the builtin types. Also simplify uniquer_hashables to
use an faster implementation under the assumption of ordered dictionaries.

From Ryan Saunders:
- Fixed runtest.py failure on Windows caused by excessive escaping of the path to python.exe.

From Lukas Schrangl:
- Run LaTeX after biber/bibtex only if necessary

From Flaviu Tamas:
- Added -fsanitize support to ParseFlags(). This will propagate to CCFLAGS and LINKFLAGS.

From Mats Wichmann:
- A list argument as the source to the Copy() action function is now
correctly handled by converting elements to string. Copy errors out
if asked to copy a list to an existing non-directory destination.
Both the implementation and the strfunction which prints the progress
message were adjusted. Fixes 3009.
- doc: EnsureSConsVersion, EnsurePythonVersion, Exit, GetLaunchDir and
SConscriptChdir are now listed as Global functions only; the
Environment versions still work but are not documented.
- The Java scanner processing of JAVACLASSPATH for dependencies was
changed to split on os.pathsep instead of space, to match usage of
passing a path string like "xxx:yyy:zzz". This is not portable -
passing a POSIX-style path string (with ':') won't work on Windows
(';') - which is now documented with a hint to use a list instead
to be portable. Splitting on space broke paths with embedded spaces.
Fixes 4243.
- Cleanup: make sure BoolVariable usage in tests and examples uses Python
boolean values instead of 0/1.
- Stop telling people to run "python setup.py install" in the User Guide.
Adds new content on using virtualenvs to be able to have multiple
different SCons versions available on one system.
- Added the "DefaultEnvironment(tools=[])" stanza to a number of tests
that are known to be particularly slow. It's still just a tiny
speedup, but the Windows CI had been occasionally timing out,
so maybe this helps a bit.
- Remove an extra existence check in one ninja test that caused it
to be skipped on some otherwise-valid Windows installations.
- test framework tests now pass on Linux and Windows (the latter can
still run into problems on some configurations), and automated
tests are now run on changes in this area so future problems can
be spotted.
- The single-file Util module was split into a package with a few
functional areas getting their own files - Util.py had grown to
over 2100 lines.
- Add a zipapp package of scons-local: can use SCons from a local
file which does not need unpacking.
- Additional explanations for MSVSProject and MSVSSolution builders.
- Fix a problem (present in 4.4.0 only) where a Java inner class could
not be cached because the emitted filename contained a '$' and when
looked up through a node ended up generating a Python SyntaxError
because it was passed through scons_subst().
- Have the gfortran tool do a better job of honoring user preferences
for the dialect tools (F77, F90, F03 and F09, as well as the shared-library
equivalents SHF77, SHF90, SHF03, SHF09). Previously these were
unconditionally overwritten to 'gfortran'; the change should be more
in line with expectations of how these variables should work.
Also cleaned a few Fortran tests - test behavior does not change.
- Updated MSVC documentation - adds "version added" annotations on recently
added construction variables and provides a version-mapping table.
- Add Python 3.12 support, and indicate 3.11/3.12 support in package.

From Joseph Brill:
- Verify that a user specified msvc script (via MSVC_USE_SCRIPT) exists and raise an exception
when the user specified msvc script does not exist.
- Fix issue where if you only had mingw installed on a Windows system and no MSVC compiler, and
did not explicitly request the mingw tool, mingw tool initialization would fail and set the
default compiler to MSVC which wasn't installed, yielding broken build.
Updated mingw tool so that the generate and exists methods use the same mingw search paths
(issue 4134).
- Update the debug output written to stdout for MSVC initialization which is enabled by setting
SCONS_MSCOMMON_DEBUG=- to use the logging module. Also changed the debug output format
written to stdout to include more information about the source for each message of MSVC
initialization debugging output. A single space was added before the message for all
debugging output records written to stdout and to files.
- Refactor the data definitions for msvc configurations to allow derived data structures to be
constructed during initialization that removes the need for special case handling during
runtime execution. Special case handling of host/target combinations is eliminated and
replaced with pre-computed search lists that implicitly handle the differences between full
versions and express versions of msvc. This fixes an issue where Express versions of the MSVC
compiler were not detected due to differences in initial msvc detection and msvc batch file
determination when configuring the build environment. This could lead to build failures when
only an MSVC Express instance is installed and the MSVC version is not explicitly specified
(issue 2668 and issue 2697).
- Added MSVC_USE_SETTINGS construction variable to pass a dictionary to configure the msvc compiler
system environment as an alternative to bypassing Visual Studio autodetection entirely.
- Added MSVC_SDK_VERSION construction variable which allows building with a specific Microsoft
SDK version. This variable is used with the msvc batch file determined via autodetection subject
to validation constraints. Refer to the documentation for additional requirements and validation
details.
- Added MSVC_TOOLSET_VERSION construction variable which allows building with a specific toolset
version. This variable is used with the msvc batch file determined via autodetection subject to
validation constraints. This variable does not affect the autodetection and selection of msvc
instances. The toolset version is applied after an msvc instance is selected. This could be the
default version of msvc. Refer to the documentation for additional requirements and validation
details. Addresses issue 3265, issue 3664, and pull request 4149.
- Added MSVC_SPECTRE_LIBS construction variable which allows building with spectre-mitigated
Visual C++ libraries. This variable is used with the msvc batch file determined via autodetection
subject to validation constraints. Refer to the documentation for additional requirements and
validation details.
- Added MSVC_SCRIPT_ARGS construction variable which specifies command line arguments that are
passed to the msvc batch file determined via autodetection subject to validation constraints.
Refer to the documentation for additional requirements and validation details. Addresses
enhancement issue 4106.
- An exception is raised when MSVC_UWP_APP is enabled for Visual Studio 2013 and earlier.
Previous behavior was to silently ignore MSVC_UWP_APP when enabled for Visual Studio 2013
and earlier. Refer to the documentation for additional requirements and validation details.
MSVC_UWP_APP was extended to accept True, False, and None in addition to '1' and '0'.
- The imported system environment variable names for MSVC 7.0 and 6.0 have been changed to the
names set by their respective installers. Prior to this change, bypassing MSVC detection by
specifying the MSVC 7.0 batch file directly would fail due to using an erroneous environment
variable name. Arguments are no longer passed to the MSVC 6.0 to 7.1 batch files as no
arguments are required and could improve the effectiveness of the internal MSVC cache.
- Propagate the OS and windir environment variables from the system environment to the msvc
environment. The OS and windir environment variables are used in the MSVC 6.0 batch file
and the SDK 6.0-7.1 SetEnv.cmd batch files. Inclusion of the OS and windir environment
variables eliminates some partial paths and warnings generated by the MSVC 6.0 and SDK
6.0-7.1 batch files when the variables are not defined.
Note: Attempting to run the SDK 6.0-7.1 batch files directly via MSVC_USE_SCRIPT can lead to
build failures and/or incomplete build environments. The SDK 6.0-7.1 batch files
require delayed expansion to be enabled which is currently not supported and is
typically not enabled by default on the host system. The batch files may also require
environment variables that are not included by default in the msvc environment.
- Suppress issuing a warning when there are no installed Visual Studio instances for the default
tools configuration (issue 2813). When msvc is the default compiler because there are no
compilers installed, a build may fail due to the cl.exe command not being recognized. At
present, there is no easy way to detect during msvc initialization if the default environment
will be used later to build a program and/or library. There is no error/warning issued for the
default tools as there are legitimate SCons uses that do not require a c compiler.
- Added a global policy setting and an environment construction variable for specifying the
action to be taken when an msvc request cannot be satisfied. The available options are "error",
"exception", "warning", "warn", "ignore", and "suppress". The global policy variable may be
set and retrieved via the functions msvc_set_notfound_policy and msvc_get_notfound_policy,
respectively. These two methods may be imported from SCons.Tool.MSCommon. The environment
construction variable is MSVC_NOTFOUND_POLICY. When defined, the environment construction
variable overrides the global policy setting for a given environment. When the active policy
is "error" or "exception", an MSVCVersionNotFound exception is raised. When the active policy
is "warning" or "warn", a VisualCMissingWarning warning is issued and the constructed
environment is likely incomplete. When the active policy is "ignore" or "suppress", no action
is taken and the constructed environment is likely incomplete. As implemented, the default
global policy is "warning". The ability to set the global policy via an SCons command-line
option may be added in a future enhancement.
- Added a global policy setting and an environment construction variable for specifying the
action to be taken when msvc script errors are detected. The available options are "error",
"exception", "warning", "warn", "ignore", and "suppress". The global policy variable may be
set and retrieved via the functions msvc_set_scripterror_policy and msvc_get_scripterror_policy,
respectively. These two methods may be imported from SCons.Tool.MSCommon. The environment
construction variable is MSVC_SCRIPTERROR_POLICY. When defined, the environment construction
variable overrides the global policy setting for a given environment. When the active policy
is "error" or "exception", an MSVCScriptExecutionError exception is raised when msvc batch file
errors are detected. When the active policy is "warning" or "warn", an MSVCScriptExecutionWarning
warning is issued when msvc batch file errors are detected. When the active policy is "ignore" or
"suppress", msvc batch error messages are suppressed. As implemented, the default global policy
is "ignore". The ability to set the global policy via an SCons command-line option may be added
in a future enhancement.
- Added experimental function msvc_query_version_toolset to SCons.Tool.MSCommon. Given a version
specification, this function will return an msvc version and an msvc toolset version. The msvc
toolset version may be None. The msvc version and msvc toolset version can be used in the
environment construction variables MSVC_VERSION and MSVC_TOOLSET_VERSION, respectively. The
version specification may be an msvc version or an msvc toolset version. This is a proxy for
using an msvc toolset version to select an msvc instance. This function may be removed when an
msvc toolset version is used during msvc instance selection.
- Modify the MSCommon logger configuration to be independent of the root logger. This fixes an issue
when multiple loggers are created and the MSCommon logger added computed fields to the root logger
that are not present in other logging instances.
- Modify the MSVC_USE_SCRIPT_ARGS test fixture to disable the msvc cache. This fixes an issue where
the MSVC_USE_SCRIPT_ARGS test for success relied on a debug log message that was not produced when
the msvc cache file exists and the test keys are already in the cache as the msvc script invocation
was bypassed.

From William Deegan:
- Fix check for unsupported Python version. It was broken. Also now the error message
will include what is the minimum supported version of Python
- Fix ActionTests to work with python 3.10.1 (and higher)
NOTE: If you build with Python 3.10.0 and then rebuild with 3.10.1 (or higher), you may
see unexpected rebuilds. This is due to Python internals changing which changed
the signature of a Python Action Function.
- Fix a number of Python ResourceWarnings which are issued when running SCons and/or it's tests
with python 3.9 (or higher)
- Action._subproc() can now be used as a python context manager to ensure that the
POpen object is properly closed.
(Thanks to Mats Wichmann for catching that DummyPopen needed additional logic)
- Added project_url for mailing lists and Discord
- Updated project url in steup.cfg to be https instead of http
- Updated setup.cfg to remove Python 3.5 and add Python 3.10
- Added default values for source and target arguments to _defines() function. This
is used to expand CPPDEFINES (and others). Previous change added those arguments
with no defaults, so old usage where _defines() was called without source and target
arguments would yield an exception. This issue was found via qt4 and qt5 tools in
scons-contrib https://github.com/SCons/scons-contrib/issues/45

From David H:
- Add JavaScanner to include JAVACLASSPATH as a dependency when using the Java tool.
- Fix incorrect Java classpath generation when a NodeList is used as part of any JAVA*PATH variables.

From Daniel Moody:
- Add cache-debug messages for push failures.
- Ninja: Changed generated build.ninja file to run SCons only build Actions via
a SCons Deamon. Added logic for starting and connecting to SCons daemon (currently
only used for ninja)
- Ninja: Fix issue where Configure files weren't being properly processed when build run
via ninja.
- Ninja: Added ninja mingw support and improved ninja CommandGeneratorAction support.
- Ninja: Update ninja file generation to only create response files for build commands
which exceed MAXLINELENGTH
- Ninja: Added NINJA_GENERATED_SOURCE_ALIAS_NAME which allows user to specify an
Alias() which the ninja tool can use to determine which files are generated sources.
If this is not set by the user then the ninja tool will still dynamically determine
which files are generated sources based on NINJA_GENERATED_SOURCE_SUFFIXES, and create
a phony target _ninja_generated_sources. Generated sources will be built first by
ninja. This is needed because ninja cannot determine which generated sources are
required by other build targets. Code contributed by MongoDB
The downstream commit is here:
https://github.com/mongodb/mongo/commit/2fef432fa6e7cf3fd4f22ba3b193222c2887f14f
- Ninja: Added special case for ninja scons daemon to work in win32 python3.6 environments.
This particular environment does a bad job managing popen standard file handles, so
some special workarounds are needed.
- Ninja:Added user configurable setting of ninja depfile format via NINJA_DEPFILE_PARSE_FORMAT.
Now setting NINJA_DEPFILE_PARSE_FORMAT to [msvc,gcc,clang] can force the ninja expected
format. Compiler tools will also configure the variable automatically.
- Ninja: Made ninja tool force the ninja file as the only target.
- Ninja: Improved the default targets setup and made sure there is always a default target for
the ninja file, which excludes targets that start and stop the daemon.
- Ninja: Update ninja tool so targets passed to SCons are propagated to ninja when scons
automatically executes ninja.
- Small refactor of scons daemons using a shared StateInfo class for communication
between the scons interactive thread and the http server thread. Added error handling
for scons interactive failing to startup.
- Ninja: Updated ninja scons daemon scripts to output errors to stderr as well as the daemon log.
- Ninja: Fix typo in ninja scons daemon startup which causes ConnectionRefusedError to not retry
- Added SHELL_ENV_GENERATORS construction variable. This variable should be set to a list
(or an iterable) which contains functions to be called in order
when constructing the execution environment (Generally this is the shell environment
variables). This allows the user to customize how (for example) PATH is constructed.
Note that these are called for every build command run by SCons. It could have considerable
performance impact if not used carefully.
to connect to the server during start up.
- lex: Fixed an issue with the lex tool where file arguments specified to either "--header-file="
or "--tables-file=" which included a space in the path to the file would be processed incorrectly
- Ninja: added option "--skip-ninja-regen" to enable skipping regeneration of the ninja file
if scons can determine the ninja file doesnot need to be regenerated, which will also
skip restarting the scons daemon. Note this option is could result in incorrect rebuilds
if scons Glob or scons generated files are used in ninja build target's command lines.
- Ninja: Added new alias "shutdown-ninja-scons-daemon" to allow ninja to shutdown the daemon.
Also added cleanup to test framework to kill ninja scons daemons and clean ip daemon logs.
NOTE: Test for this requires python psutil module. It will be skipped if not present.
- Ninja: Added command line variable NINJA_CMD_ARGS that allows to pass through ninja command line args.
This can also be set in your Environment().

From Mats Wichmann:
- Tweak the way default site_scons paths on Windows are expressed to
conform to conventions (what they actually resolve to is unchanged),
drop a Py2 workaround, and pick a better "system" path, old one
remains supported (%AllUsersProfile%\scons\site_scons vs old
%AllUsersProfile%\Application Data\scons\site_scons).
- Fix testsuite to work on Windows systems where there is no usable
association for running .py files directly. There are a few tests where
we need to do this for internal reasons, those are skipped in that case.
Bad association could mean some other tool took it over (Visual
Studio Code is known to do this), or no association at all.
- Updated debug code in MSVC and MSVS tools to conform to the
suggested "lazy interpolation" use of the Python logging module.
Calls now look like 'debug("template %s", text)' rather than
'debug("template %s" % text)' so the logging system does the
interpolation only when/if needed (was a pylint warning).
- Update Help (-H) output a bit. Drop "ignored for compat" entry.
Pass window size to formatter so it formats for wider displays too.
- runtest.py now accepts -j 0 to auto-detect number of usable
processors for testing threads.
- Fixed crash in C scanner's dictify_CPPDEFINES() function which happens if
AppendUnique is called on CPPPATH. (Issue 4108).
- The MSVC script_env_cache now contains a sanity check: if the retrieved
tools path does not exist, the entry is invalidated so it will
be recomputed, in an attempt to avoid scons failing when certain
compiler version bumps have taken place. The dictionary key (uses
the name of a batch file and any arguments which may have been
passes), is now computed a bit differently: the dashes are left
off if there are no arguments. The default cachefile is changed
to have a .json suffix, for better recognition on Windows since
the contents are json.
- As "code modernization" all of SCons now uses the current super()
zero-argument syntax instead of direct calls to a parent class method
or the super() two-argument syntax.
- Renamed ParseFlag's internal data structure to "mapping" instead of
"dict" (avoid redefining builtin)
- Fix an old use-before-set bug in tex tool (issue 2888)
- Fix a test harness exception returning stderr if a wait_for timed out.
- ParseConfig now correctly passes the *unique* flag to a user-supplied
flag-merging function.
- Restore the ability of the content-timestamp decider to see that a
a source which is a symlink has changed if the file-system target of
that link has been modified (issue 3880)
- Modernize a few tests that use now-deprecated unittest.getTestCaseNames
and unittest.makeSuite - Python itself suggests the replacements.
- SCons.Tool.find_program_path now takes an optional add_path argument
to add a path to the execution environment if it was discovered in
default_paths. Previously, the routine, called by many tool modules,
never altered the execution environment, leaving it to the tools.
- A new construction variable FORTRANCOMMONFLAGS is added which is
applied to all Fortran dialects, in case someone needs to set some
flags globally. FORTRANFLAGS looked like it was intended for that,
but was not applied to other dialects, and e2e tests explicitly checked
that FORTRANFLAGS did not propagate outside the FORTRAN dialect,
so the conclusion is that behavior is intentional (issue 2257)
- SCons programmatic importing (tool modules and platform modules)
no longer uses the deprecated (since Py 3.10) importlib.load_module
routine, shifting to the preferred exec_module. Old Python 2 compatible
import fallback (using the imp module) in tool module loading is dropped.
Tool module loading no longer special-cases Jython, which is a dead
project as far as SCons (no timeline in sight for Python 3 support).
- Improvements to lex and yacc tools: better documentation of
extra-file options, add test for extra-file behavior.
- Two new construction variables are introduced for lex (LEX_HEADER_FILE
and LEX_TABLES_FILE) as the preferred way of specifying these extra-file
options.
- Two new construction variables are introduced for yacc
(YACC_HEADER_FILE and YACC_GRAPH_FILE) as the preferred way of
specifying these extra-file options.


From Zhichang Yu:
- Added MSVC_USE_SCRIPT_ARGS variable to pass arguments to MSVC_USE_SCRIPT.
- Added Configure.CheckMember() checker to check if struct/class has the specified member.

From Ivan Kravets, PlatformIO:
- Conditional C/C++ Preprocessor: Strip shell's backslashes from the computed include (-DFOO_H=\"foo.h\")

From Jacob Cassagnol:
- Default hash algorithm check updated for SCons FIPS compliance. Now checks for hash viability
first and then walks the tree to use the first viable hash as the default one. This typically
selects SHA1 on FIPS-enabled systems less than Python 3.9 as the new default instead of MD5,
unless SHA1 has also been disabled by security policy, at which point SCons selects SHA256
as the default. For systems running Python 3.9 and later, the hashlib bug has been fixed,
and SCons will once again default to MD5 as the preferred algorithm.

From Joseph Brill:
- Fix MSVS tests (vs-N.N-exec.py) for MSVS 6.0, 7.0, and 7.1 (import missing module).
- Add support for Visual Studio 2022.

From William Deegan:
- Fix reproducible builds. Restore logic respecting SOURCE_DATE_EPOCH when set.
- Fix version tests to work with updated scons --version output. (Date format changed)
- Fix issue 4021. Change the way subst() is used in Textfile() to not evaluate '$$(' -> '$',
but instead it should yield '$('.
- Change SCons.Platform.win32.get_architecture() to return platform.platform() when run in an
environment where neither: PROCESSOR_ARCHITEW6432 nor PROCESSOR_ARCHITECTURE is set.
This should fix platform tests which started failing when HOST_OS/HOST_ARCH changes
introduced by Aaron Franke (listed below) were merged.
- Further PCH updates. It's now recommended that env['PCH'] should always be a File node.
Either via return value from env.PCH() or by explicitly using File('StdAfx.pch').
- Added --no-ignore-skips to runtest.py. Changed default to ignore skips when setting
runtest.py's exit status. Previously would exit 2 if any tests were skipped.
Now will only exit 2 if user specifies --no-ignore-skips and some tests were skipped.

From Ryan Egesdahl:
- Small fix to ensure CLVar default value is an empty list.
See MongoDB bug report: https://jira.mongodb.org/browse/SERVER-59656
Code contributed by MongoDB.
- Ninja - Fixed an issue where if you control-c and/or killed ninja while it was running scons to
regenerate build.ninja you would end up with no build.ninja file and have to rerun scons from scratch.
Code contributed by MongoDB.

From Aaron Franke:
- Define HOST_OS and HOST_ARCH in the environment for all platforms.
Before this change, these were only defined for Win32 and OS/2.

From Daniel Moody:
- Fix ninja tool to never use for_sig substitution because ninja does not use signatures. This
issue affected CommandGeneratorAction function actions specifically.
- Expanded ninja Mkdir to also support Mkdir actions.
- Added support for the PCH environment variable to support subst generators.
- Fix command line escaping for ninja dollar sign escape. Without escaping ninja properly,
the ninja file scons regenerate and callback invocations will lose the $ characters used in
the scons command line which ninja uses itself for escaping. For Example:
scons BUILD=xyz OTHERVAR=$BUILD
Prior to this fix, it would cause ninja to fail to escape the dollar sign, leading to the
single dollar sign being used as a ninja escape character in the ninja file.

From Daniel Moody:
- Added ninja API 'NINJA_FORCE_SCONS_BUILD' to force a node to callback to scons.

From Mats Wichmann:
- Two small Python 3.10 fixes: one more docstring turned into raw
because it contained an escape; updated "helpful" syntax error message
from 3.10 was not expected by SubstTests.py and test/Subst/Syntax.py
- EmitterProxy rich comparison set is completed (checker warning).
Added __le__, __gt__, __ge__.
- Fix gcc/g++ tool failing if "gcc --version" returns text which fails
to_String conversion (i.e., not UTF-8) - failure happens when tool
initialization checks version. For gcc, the initial version string is
not translated, for the rest, don't convert, just consume raw and discard.
- Maintenance and doc: modernize some usage in Scanner package,
calling super(), switching some imitialization to comprehensions,
and code formatting. Docstring for scanner Base moved from
init-method to class-level so it's picked up by Sphinx.
Added new sconsign filenames to skip_entry_list in Scanner/Dir.py
- Change SCons.Scanner.Base to ScannerBase. Old name kept as an alias
but is now unused in SCons itself.
- Call Variables option converter consistently - the converter should
have access to the env if it needs to (issue 2064).
- Fixed the variables Add() method to accept a tuple for the variable
name the same way AddVariables() does (issue 3869).
- The premade validator PathIsDirCreate for for PathVariable now catches
the case where the directory could not be created due to permission
problems, allowing a more helpful error to be emitted (issue 2828)
- Maintenance: Python thread.setDaemon is deprecated in favor of
directly updating daemon attribute - update SCons to do this.
- Make sure when subst'ing a callable, the callable is called with
the correct for_signature value, previously it would be true even
if doing SUBST_RAW (issue 4037)
- Update Util/NodeList implementation to get rid of a workaround for
early Python 3 slicing issue that is no longer a problem.
- Rework some Java tests to skip rather than fail on CI systems, where
the working java is > v9, but a 1.8 or 9 was also found.
- Java updates: on Windows, detect more default JDK install locations.
On all platforms, more Java versions (up to 17.0 now). Add more information
on version selection to docs.
Update docs on JavaH tool in light of javah command dropped since 10.0.
Try to be better about preserving user's passed-in JAVA* construction vars.
- Start the deprecation of the qt tool, which refers to Qt3 (usupported
since around 2006). There's a deprecation warning added, initially
defaulting to disabled.

From Brian Quistorff:
- Fix crash when scons is run from a python environement where a signal
is set from outside Python.

From Byron Platt:
- Fix Install() issue when copytree recursion gives bad arguments that can
lead to install side-effects including keeping dangling symlinks and
silently failing to copy directories (and their subdirectories) when the
directory already exists in the target.

From Joseph Brill:
- Internal MSVS update: Remove unnecessary calls to find all installed versions of msvc
when constructing the installed visual studios list.

From William Deegan:
- Improve Subst()'s logic to check for proper callable function or class's argument list.
It will now allow callables with expected args, and any extra args as long as they
have default arguments. Additionally functions with no defaults for extra arguments
as long as they are set using functools.partial to create a new callable which set them.
- Fix Issue 3035 - mingw with SHLIBVERSION set fails with either not a dll error or
"Multiple ways to build the same target were specified for:". Now mingw will disable
creating the symlinks (and adding version string to ) dlls. It sets SHLIBNOVERSIONSYMLINKS,
IMPLIBNOVERSIONSYMLINKS and LDMODULENOVERSIONSYMLINKS to True.
- Added --experimental flag, to enable various experimental features/tools. You can specify
'all', 'none', or any combination of available experimental features.
- Fix Issue 3933 - Remove unguarded print of debug information in SharedLibrary logic when
SHLIBVERSION is specified.
- Fix versioned shared library naming for MacOS platform. (Previously was libxyz.dylib.1.2.3,
has been fixed to libxyz.1.2.3.dylib. Additionally the sonamed symlink had the same issue,
that is now resolved as well)
- Add experimental ninja builder. (Contributed by MongoDB, Daniel Moody and many others).
- Fix 3955 - _LIBDIRFLAGS leaving $( and $) in *COMSTR output. Added affect_signature flag to
_concat function. If set to False, it will prepend and append $( and $). That way the various
Environment variables can use that rather than "$( _concat(...) $)".
- Fix issue with exparimental ninja tool which would fail on windows or when ninja package wasn't
installed but --experimental=ninja was specified.
- As part of experimental ninja tool, allow SetOption() to set both disable_execute_ninja and
disable_ninja.

From David H:
- Fix Issue 3906 - `IMPLICIT_COMMAND_DEPENDENCIES` was not properly disabled when
set to any string value (For example ['none','false','no','off'])
Also previously 'All' wouldn't have the desired affect.

From Ivan Kravets:
- Provide a custom argument escape function for `TempFileMunge` using a new
`TEMPFILEARGESCFUNC` variable. Useful if you need to apply extra operations on
a command argument before writing to a temporary file (fix Windows slashes,
normalize paths, etc.)

From Henrik Maier:
- DocbookXslt tool: The XSLT stylesheet file is now initialized to an env.File() Node,
such that dependencies work correctly in hierarchical builds (eg when using
DocbookXslt in SConscript('subdir/SConscript') context.

From Daniel Moody:
- Update CacheDir to use uuid for tmpfile uniqueness instead of pid.
This fixes cases for shared cache where two systems write to the same
cache tmpfile at the same time because the happened to get the same pid.
- Added support for passing custom CacheDir derived classes to SCons. Moved
copy_from_cache attribute from the Environment class to CacheDir class.
Code contributed by MongoDB.
- Update BuildTask to pass all targets to the progress object fixing an issue
where multi-target build nodes only got the first target passed to the progress
object.
- Fix a potential race condition in shared cache environments where the permissions are
not writeable for a moment after the file has been renamed and other builds (users) will copy
it out of the cache. Small reorganization of logic to copy files from cachedir. Moved CacheDir
writeable permission code for copy to cache behind the atomic rename operation.
- Added marking of intermediate and and multi target nodes generated from SConf tests so that
is_conftest() is more accurate.
- Added test for configure check failing to ensure it didn't break generating and running ninja.


From Mats Wichmann:
- Initial support in tests for Python 3.10 - expected bytecode and
one changed expected exception message. Change some more regexes
to be specified as rawstrings in response to DeprecationWarnings.
- Add an example of adding an emitter to User Guide (concept
from Jeremy Elson)
- Add timing information for sconsign database dump when --debug=time
is selected. Also switch to generally using time.perf_counter,
which is the Python recommended way for timing short durations.
- Drop remaining definitions of dict-like has_key methods, since
Python 3 doesn't have a dictionary has_key (maintenance)
- Do not treat --site-dir=DIR and --no-site-dir as distinct options.
Allows a later instance to override an earlier one.
- Ignore empty cmdline arguments when computing targets (issue 2986)
- Remove long-deprecated construction variables PDFCOM, WIN32_INSERT_DEF,
WIN32DEFPREFIX, WIN32DEFSUFFIX, WIN32EXPPREFIX, WIN32EXPSUFFIX.
All have been replaced by other names since at least 1.0.
- Add a __iadd__ method to the CLVar class so that inplace adds
(+=) also work as expected (issue 2399)
- Remove local copy of CLVar in EnvironmentTests unittest file -
should be testing against the production version, and they
didn't really differ.
- Don't strip spaces in INSTALLSTR by using raw subst (issue 2018)
- Deprecate Python 3.5 as a supported version.
- CPPDEFINES now expands construction variable references (issue 2363)
- Restore behavior that Install()'d files are writable (issue 3927)
- Simplified Mkdir(), the internal mkdir_func no longer needs to handle
existing directories, it can now pass exist_ok=True to os.makedirs().
- Avoid WhereIs exception if user set a tool name to empty (from issue 1742)
- Maintenance: remove obsolete __getslice__ definitions (Py3 never calls);
add Node.fs.scandir to call new (Py3.5) os.scandir; Node.fs.makedirs
now passes the exist_ok flag; Cachedir creation now uses this flag.
- Maintenance: remove unneeded imports and reorganize some. Fix uses
of warnings in some tools which instantiated the class but did nothing
with them, need to instead call SCons.Warnings.warn with the warn class.
- Drop overridden changed_since_last_build method in Value class.
- Resync the SetOption implementation and the manpage, making sure new
options are available and adding a notes column for misc information.
SetOption equivalents to --hash-chunksize, --implicit-deps-unchanged
and --implicit-deps-changed are enabled.
- Add tests for SetOption failing on disallowed options and value types.
- Maintenance: eliminate lots of checker complaints about Util.py.
- Maintenance: fix checker-spotted issues in Environment (apply_tools)
and EnvironmentTests (asserts comparing with self).
For consistency, env.Tool() now returns a tool object the same way
Tool() has done.
- Change SConscript() missing SConscript behavior - if must_exist=False,
the warning is suppressed.
- Make sure TEMPFILEPREFIX can be set to an empty string (issue 3964)

From Dillan Mills:
- Add support for the (TARGET,SOURCE,TARGETS,SOURCES,CHANGED_TARGETS,CHANGED_SOURCES}.relpath property.
This will provide a path relative to the top of the build tree (where the SConstruct is located)
Fixes 396

From Andrew Morrow:
- Fix issue 3790: Generators in CPPDEFINES now have access to populated source
and target lists

From James Benton:
- Add COMPILATIONDB_PATH_FILTER env option for CompilationDatabase() builder which allows
filtering of entries based on the output file paths using glob style file matching (issue 3742).

From Joseph Brill:
- Internal MSVC and test updates: Rework the msvc installed versions cache so that it
is not exposed externally and update external references accordingly.
- Modify the MSCommon internal-use only debug logging records to contain the correct relative
file path when the debug function is called from outside the MSCommon module.

From William Deegan:
- Fix yacc tool, not respecting YACC set at time of tool initialization.
- Refactor SCons.Tool to move all common shared and loadable module linking logic to SCons.Tool.linkCommon
- Remove pywin32 imports from SCons.Script.Main. No longer needed.
- Switch to use ctypes instead of pywin32 (requiring an extra pip install) - Fixes Github Issue 2291
- pywin32 no longer necessary for SCons install. (pip install SCons will no longer also require pywin32 on win32)
- Remove pywin32 usage from SCons.Util where it was used for accessing the registry. Python native winreg
library already includes this functionality.
- Remove using pywin32 to retrieve peak memory usage on Win32 for `--debug=memory`
- Fix Issue 3759 - include scons.1, sconsign.1, scons-time.1 manpages in sdist and wheel packages.
- Change SCons's build so the generated `SCons/__init__.py` is no longer removed by `scons -c`
- Completely rewrote versioned shared libraries logic. Added support for SOVERSION via dmoody's initial PR 3733
- No longer automatically disable setting SONAME on shared libraries on OpenBSD.
- Fix race condition bug when initializing a scons cache directory at the
same time from multiple threads or processes. Problem described in PR 3114.
This is a simpler fix which should avoid some problems identified with the initial PR.
(Credit to Fredrik Medley for reporting the issue, the initial PR, and discussing and testing
this solution)
- Minor edits to User Guide and manpage's header with copyright, released date changed.

From Michał Górny:
- Fix dvipdf test failure due to passing incorrect flag to dvipdf.

From Adam Gross:
- Fix minor bug affecting SCons.Node.FS.File.get_csig()'s usage of the MD5 chunksize.
User-facing behavior does not change with this fix (GH Issue 3726).
- Fix occasional test failures caused by not being able to find a file or directory fixture
when running multiple tests with multiple jobs.
- Added support for a new command-line parameter `--hash-format` to override the default
hash format that SCons uses. It can also be set via `SetOption('hash_format')`. Supported
values are: `md5`, `sha1`, and `sha256`. For all hash formats other than
the default of `md5`, the SConsign database will include the name of the hash format.
For example, `--hash-format=sha256` will create a SConsign with name
`.sconsign_sha256.dblite.`.
- Fix incorrect cache hits and/or misses when running in interactive mode by having
SCons.Node.Node.clear() clear out all caching-related state.
- Change Environment.SideEffect() to not add duplicate side effects.
NOTE: The list of returned side effect Nodes will not include any duplicate side effect Nodes.
- Add support to the Python scanner for finding dynamically generated dependencies.
Previously the scanner only found imports if they existed on disk at scanning time.

From David H:
- Add ZIP_OVERRIDE_TIMESTAMP env option to Zip builder which allows for overriding of the file
modification times in the archive.
- Fix Zip builder not rebuilding when ZIPROOT env option was changed.

From Jason Kenny
- Fix python3 crash when Value node get_text_content when child content does not have decode()
NOTE: If you depend on Value node's get_text_content returning concatenated contents of it's
children. This may break your code. It now concatenates the csig() of all children.

From Joachim Kuebart:
- Suppress missing SConscript deprecation warning if `must_exist=False`
is used.

From Rocco Matano:
- Fix Zip tool to respect ZIPCOMSTR. Previously all zip builder calls would yield something
like zip(["test.zip"], ["zip_scons.py"]) and ignore ZIPCOMSTR if ZIPCOM and ZIPCOMSTR
weren't set after the Environment/Tool is initialized. (Explained in PR 3659)

From Daniel Moody:
- Fix issue where java parsed a class incorrectly from lambdas used after a new.

From Simon Tegelid
- Fix using TEMPFILE in multiple actions in an action list. Previously a builder, or command
with an action list like this:
['${TEMPFILE("xxx.py -otempfile $SOURCE")}', '${TEMPFILE("yyy.py -o$TARGET tempfile")}']
Could yield a single tempfile with the first TEMPFILE's contents, used by both steps
in the action list.

From Mats Wichmann:
- Complete tests for Dictionary, env.keys() and env.values() for
OverrideEnvironment. Enable env.setdefault() method, add tests.
- Raise an error if an option (not otherwise consumed) is used which
looks like an abbreviation of one one added by AddOption. (3653)
- Tool module not found will now raise a UserError to more clearly indicate this is
probably an SConscript problem, and to make the traceback more relevant.
- Fix three issues with MergeFlags:
- Signature/return did not match documentation or existing usage - the implementation
now no longer returns the passed env
- merging --param arguments did not work (issue 3107);
- passing a dict to merge where the values are strings failed (issue 2961).
- Include previously-excluded SideEffect section in User Guide.
- Clean up unneeded imports (autoflake tool).
- Make sure cProfile is used if profiling - SCons was expecting
the Util module to monkeypatch in cProfile as profile if available,
but this is no longer being done.
- Cleanup in SCons.Util.AddMethod. If called with an environment instance
as the object to modify, the method would not be correctly set up in
any Clone of that instance. Now tries to detect this and calls
MethodWrapper to set up the method the same way env.AddMethod does.
MethodWrapper moved to Util to avoid a circular import. Fixes 3028.
- Some Python 2 compatibility code dropped
- Rework runtest.py to use argparse for arg handling (was a mix
of hand-coded and optparse, with a stated intent to "gradually port").
- Add options to runtest to generate/not generate a log of failed tests,
and to rerun such tests. Useful when an error cascades through several
tests, can quickly try if a change improves all the fails. Dropped
runtest test for fallback from qmtest, not needed; added new tests.
- Eliminate tex tool usage of "for foo in range(len(iterable))"
- Restore internal Trace function to functional state.
- Only try to initialize the wix tool by default (or when tool `default` is explicitly installed)
on Windows based systems.
- Pick a better "Topic" Trove classifier for SCons: SW Dev / Build Tools
- Use os.replace instead of os.rename in dblite so don't need to
special-case Windows here. dblite is the default storage engine for the SConsign file(s).
- Fix cut-n-paste error in msvc debug printout and make some debug output
in msvs and msvsTests.py be off until needed (uncomment to use)
- Fix Issue 3014 - Empty file and missing file have same csig
- Refactor env.Append/Prepend to remove Py 1.5 era need to nest
try blocks, can now "continue" at the appropriate places.
- Add /snap/bin to env['PATH'] on POSIX, although this is only
really useful for a subset of POSIX systems that use snaps.
Was needed for CI builds, which run on Ubuntu LTS images.
- Eliminate Py2-ism __nonzero__ (now __bool__). Work around issue 3860
where a check for BuilderBase raising exc. on __bool__ was optimized out.
This issue was found due to a bug in Python 3.10.0a4. See issue 3860 for details.