Scancode-toolkit

--------

This is a major version with no breaking API changes. Heads-up: the next version
will bring up some significant API changes summarized above.


Security:
~~~~~~~~~

- Update dependency versions for security fixes.


License scanning:
~~~~~~~~~~~~~~~~~

- Add 22 new licenses and update 71 existing licenses

- Update licenses to include the SPDX license list 3.12

- Improve license detection accuracy with over 2,300 new and updated license
detection rules

- Undeprecate the regexp license and deprecate the hs-regexp-orig license

- Improve license db initial load time with caching for faster scancode
start time

- Add experimental SCANCODE_LICENSE_INDEX_CACHE environment variable to point
to an alternative directory where the license index cache is stored (as
opposed to store this as package data.)

- Ensure that license short names are not more than 50 characters long

- Thank you to:
- Dennis Clark DennisClark
- Chin-Yeung Li chinyeungli
- Armijn Hemmel armijnhemel
- Sarita Singh itssingh
- Akanksha Garg akugarg


Copyright scanning:
~~~~~~~~~~~~~~~~~~~

- Detect SPDX-FileCopyrightText as defined by the FSFE Reuse project
Thank you to Daniel Eder daniel-eder

- Fix bug when using the --filter-clues command line option
Thank you to Van Lindberg VanL

- Fixed copyright truncation bug
Thank you to Akanksha Garg akugarg


Package scanning:
~~~~~~~~~~~~~~~~~

- Add support for installed RPMs detection internally (not wired to scans)
Thank you to Chin-Yeung Li chinyeungli

- Improve handling of Debian copyright files with faster and more
accurate license detection
Thank you to Thomas Druez tdruez

- Add new built-in support for installed_files report. Only available when
used as a library.

- Improve support for RPM, npm, Debian, build scripts (Bazel) and Go packages
Thank you to:
- Divyansh Sharma Divyansh2512
- Jonothan Yang JonoYang
- Steven Esser majurg

- Add new support to collect information from semi-structured Readme files
and related metadata files.
Thank you to Jonothan Yang JonoYang and Steven Esser majurg


Outputs:
~~~~~~~~~

- Add new Debian copyright-formatted output.
Thank you to Jelmer Vernooij jelmer

- Fix bug in --include where directories where not skipped correctly
Thank you to Pierre Tardy tardyp


Misc. and documentation improvements:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Update the way tests assertions are made
Thank you to Aditya Viki adityaviki

- Thank you to Aryan Kenchappagol aryanxk02

--------

Installation:
~~~~~~~~~~~~~

- Resolve reported installation issues on macOS, Windows and Linux
- Stop using extras for a default wheel installation
- Build new scancode-toolkit-mini package with limited dependencies for use
when packaging in distros and similar
- The new Dockerfile will create smaller images and containers.
Thank you to Viktor Tiulpin tiulpin

License scanning:
~~~~~~~~~~~~~~~~~

- Over 150 new and updated licenses
- Support the latest SPDX license list v3.11
- Improve license detection accuracy with over 740 new and improved license
detection rules
- Fix license cache handling issues

Misc.:
~~~~~~
- Update extractcode, typecode and their native dependencies for better support
of latests versions of macOS.

-------

Security:
~~~~~~~~~

- Update vulnerable LXML to version 4.6.2 to fix
https://nvd.nist.gov/vuln/detail/CVE-2020-27783
This was detected thanks to https://github.com/nexb/vulnerablecode

Operating system support:
~~~~~~~~~~~~~~~~~~~~~~~~~

- Drop support for Python 2 295
- Drop support for 32 bits on Windows 335
- Add support for Python 64 bits on Windows 64 bits 335
- Add support for Python 3.6, 37, 3.8 and 3.9 on Linux, Windows and macOS.
These are now tested on Azure.
- Add deprecation message for native Windows support 2366

License scanning:
~~~~~~~~~~~~~~~~~

- Improve license detection accuracy with over 8400 new license detection rules
added or updated
- Remove the previously deprecated --license-diag option
- Include pre-built license index in release archives to speed up start 988
- Use SPDX LicenseRef-scancode namespace for all licenses keys not in SPDX
- Replace DEJACODE_LICENSE_URL with SCANCODE_LICENSEDB_URL at
https://scancode-licensedb.aboutcode.org #2165
- Add new license flag in license detection results "is_license_intro" that
is used to indicate that a license rule is a short license introduction
statement (that typically may be reported as some unknown license)

Package scanning:
~~~~~~~~~~~~~~~~~

- Add detection of package-installed files
- Add analysis of system package installed databases for Debian, OpenWRT and
Alpine Linux packages
- Add support for Alpine Linux, Debian, OpenWRT.

Copyright scanning:
~~~~~~~~~~~~~~~~~~~

- Improve detection with minor grammar fixes

Misc.:
~~~~~~

- Adopt a new calendar date-based versioning for scancode-toolkit version numbers
- Update thirdparty dependencies and built-in plugins
- Allow installation without extractcode and typecode native plugins. Instead
one can elect to install these or not to have a lighter footprint if needed.
- Update configuration and bootstrap scripts to support a new PyPI-like
repository at https://thirdparty.aboutcode.org/pypi/
- Create new release scripts to populate released archives with just the
required wheels of a given OS and Python version.
- Updated scancode.bat to handle % signs in the arguments 1876

-------------------

Notable changes:
~~~~~~~~~~~~~~~~

- Collect Windows executable metadata 652
- Fix minor bugs
- Add Dockerfile to build docker image from ScanCode sources 2265

----------------------

Notable changes:
~~~~~~~~~~~~~~~~

- Use commoncode, typecode and extractcode as external standalone packages 2233

----------------------

Minor bug fixes:
~~~~~~~~~~~~~~~~

- Do not fail if Debian status is missing 2224
- Report correct detected license text in binary 2226 2227