Scancodeio

--------------------

WARNING: After upgrading the ScanCode.io codebase to this version,
and following the ``docker compose build``,
the permissions of the ``/var/scancodeio/`` directory of the Docker volumes require
to be updated for the new ``app`` user, using:
``docker compose run -u 0:0 web chown -R app:app /var/scancodeio/``

- Run Docker as non-root user using virtualenv.
WARNING: The permissions of the ``/var/scancodeio/`` directory in the Docker volumes
require to be updated for the new ``app`` user.
https://github.com/nexB/scancode.io/issues/399

- Add column sort and filters in dependency list view.
https://github.com/nexB/scancode.io/issues/823

- Add a new ``ScanCodebasePackage`` pipeline to scan a codebase for packages only.
https://github.com/nexB/scancode.io/issues/815

- Add new ``outputs`` REST API action that list projects output files including an URL
to download the file.
https://github.com/nexB/scancode.io/issues/678

- Add support for multiple to/from input files in the ``deploy_to_develop`` pipeline.
https://github.com/nexB/scancode.io/issues/813

- Add the ability to delete and download project inputs.
Note that the inputs cannot be modified (added or deleted) once a pipeline run as
started on the project.
https://github.com/nexB/scancode.io/issues/813

- Fix root_filesystem data structure stored on the Project ``extra_data`` field.
This was causing a conflict with the expected docker images data structure
when generating an XLSX output.
https://github.com/nexB/scancode.io/issues/824

- Fix the SPDX output to include missing detailed license texts for LicenseRef.
Add ``licensedb_url`` and ``scancode_url`` to the SPDX ``ExtractedLicensingInfo``
``seeAlsos``.
Include the ``Package.notice_text`` as the SPDX ``attribution_texts``.
https://github.com/nexB/scancode.io/issues/841

--------------------

- Add support for license policies and complaince alert for Discovered Packages.
https://github.com/nexB/scancode.io/issues/151

- Refine the details views and tabs:
- Add a "Relations" tab in the Resource details view
- Disable empty tabs by default
- Display the count of items in the tab label
- Improve query performances for details views
https://github.com/nexB/scancode.io/issues/799

- Upgrade vulnerablecode integration:
- Add ``affected_by_vulnerabilities`` field on ``DiscoveredPackage`` model.
- Add UI for showing package vulnerabilities in details view.
- Add packages filtering by ``is_vulnerable``.
- Include vulnerability data in the JSON results.
https://github.com/nexB/scancode.io/issues/600

- Add multiple new filtering option to list views table headers.
Refactored the way to define filters using the table_columns view attribute.
https://github.com/nexB/scancode.io/issues/216
https://github.com/nexB/scancode.io/issues/580
https://github.com/nexB/scancode.io/issues/506

- Update the CycloneDX BOM download file extension from ``.bom.json`` to ``.cdx.json``.
https://github.com/nexB/scancode.io/issues/785

- SPDX download BOM do not include codebase resource files by default anymore.
https://github.com/nexB/scancode.io/issues/785

- Add archive_location to the LAYERS worksheet of XLSX output.
https://github.com/nexB/scancode.io/issues/773

- Add "New Project" button to Project details view.
https://github.com/nexB/scancode.io/issues/763

- Display image type files in the codebase resource details view in a new "Image" tab.

- Add ``slug`` field on the Project model. That field is used in URLs instead of the
``uuid``.
https://github.com/nexB/scancode.io/issues/745

- Fix the ordering of the Codebase panel in the Project details view.
https://github.com/nexB/scancode.io/issues/795

- Do not rely on the internal ``id`` PK for package and dependency details URLs.
Package details URL is now based on ``uuid`` and the dependency details URL is based
on ``dependency_uid``.
https://github.com/nexB/scancode.io/issues/331

- Add a "License score" project setting that can be used to limit the returned license
matches with a score above the provided one.
This is leveraging the ScanCode-toolkit ``--license-score`` option, see:
https://scancode-toolkit.readthedocs.io/en/stable/cli-reference/basic-options.html#license-score-option
https://github.com/nexB/scancode.io/issues/335

- Adds a project settings ``scan_max_file_size`` and a scancode.io settings field
``SCANCODEIO_SCAN_MAX_FILE_SIZE`` to skip scanning files above a certain
file size (in bytes) as a temporary fix for large memory spikes while
scanning for licenses in certain large files.
https://github.com/aboutcode-org/scancode-toolkit/issues/3711

--------------------

- Upgrade ScanCode-toolkit to latest v32.0.x
Warning: This upgrade requires schema and data migrations (both included).
It is recommended to reset and re-run the pipelines to benefit from the latest
ScanCode detection improvements.
Refer to https://github.com/nexB/scancode-toolkit/blob/develop/CHANGELOG.rst#v3200-next-roadmap
for the full list of changes.
https://github.com/nexB/scancode.io/issues/569

- Add a new ``deploy_to_develop`` pipeline specialized in creating relations between
the development source code and binaries or deployed code.
This pipeline is expecting 2 archive files with "from-" and "to-" filename prefixes
as inputs:
1. "from-[FILENAME]" archive containing the development source code
2. "to-[FILENAME]" archive containing the deployment compiled code
https://github.com/nexB/scancode.io/issues/659

- Add ability to configure a Project through a new "Settings" form in the UI or by
providing a ".scancode-config.yml" configuration file as one of the Project inputs.
The "Settings" form allows to rename a Project, add and edit the notes, as well
as providing a list of patterns to be ignored during pipeline runs, the choice of
extracting archives recursively, and the ability to provide a custom template for
attribution.
https://github.com/nexB/scancode.io/issues/685
https://github.com/nexB/scancode.io/issues/764

- Add ``notes`` field on the Project model. Notes can be updated from the Project
settings form. Also, notes can be provided while creating a project through the CLI
using the a new ``--notes`` option.
https://github.com/nexB/scancode.io/issues/709

- Add a mapper function to relate .ABOUT files during the d2d pipeline.
https://github.com/nexB/scancode.io/issues/740

- Enhance the file viewer UI of the resource details view.
A new search for the file content was added.
Also, it is now possible to expand the file viewer in full screen mode.
https://github.com/nexB/scancode.io/issues/724

- Refine the breadcrumb UI for details view.
https://github.com/nexB/scancode.io/issues/717

- Move the "Resources status" panel from the run modal to the project details view.
https://github.com/nexB/scancode.io/issues/370

- Improve the speed of Project ``reset`` and ``delete`` using the _raw_delete model API.
https://github.com/nexB/scancode.io/issues/729

- Specify ``update_fields`` during each ``save()`` related to Run tasks,
to force a SQL UPDATE in order to avoid any data loss when the model fields are
updated during the task execution.
https://github.com/nexB/scancode.io/issues/726

- Add support for XLSX input in the ``load_inventory`` pipeline.
https://github.com/nexB/scancode.io/issues/735

- Add support for unknown licenses in attribution output.
https://github.com/nexB/scancode.io/issues/749

- Add ``License`` objects to each of the package for attribution generation.
https://github.com/nexB/scancode.io/issues/775

- The "Codebase" panel can now be used to browse the Project's codebase/ directory
and open related resources details view.
https://github.com/nexB/scancode.io/issues/744

--------------------

- Enhance the ``update_or_create_package`` pipe and add the ability to assign multiple
codebase resources at once.
https://github.com/nexB/scancode.io/issues/681

- Add new command line option to create-project and add-input management commands to
copy the content of a local source directory to the project codebase work directory.
https://github.com/nexB/scancode.io/pull/672

- Include the ScanCode-toolkit version in the output headers.
https://github.com/nexB/scancode.io/pull/670

- Enhance the ``output`` management command to support providing multiple formats at
once.
https://github.com/nexB/scancode.io/issues/646

- Improve the resolution of CycloneDX BOM and SPDX document when the file extension is
simply ``.json``.
https://github.com/nexB/scancode.io/pull/688

- Add support for manifest types using ScanCode-toolkit handlers.
https://github.com/nexB/scancode.io/issues/658

- Enhance the Resource details view to use the tabset system and display all
available data including the content viewer.
https://github.com/nexB/scancode.io/issues/215

- Add a "layers" data sheet in the xlsx output for docker pipeline run.
https://github.com/nexB/scancode.io/issues/578

- Move the ``cyclonedx`` and ``spdx`` root modules into the ``pipes`` module.
https://github.com/nexB/scancode.io/issues/657

- Remove the admin app and views.
https://github.com/nexB/scancode.io/issues/645

- Enhance the ``resolve_about_packages`` pipe to handle filename and checksum values.

- Split the pipes unit tests into their own related submodule.

- Upgrade ScanCode Toolkit to v31.2.6
https://github.com/nexB/scancode.io/issues/693