Scancodeio

--------------------

- Rename multiple pipelines for consistency and precision:
* docker: analyze_docker_image
* root_filesystems: analyze_root_filesystem_or_vm_image
* docker_windows: analyze_windows_docker_image
* inspect_manifest: inspect_packages
* deploy_to_develop: map_deploy_to_develop
* scan_package: scan_single_package

A data migration is included to facilitate the migration of existing data.
Only the new names are available in the web UI but the REST API and CLI are backward
compatible with the old names.
https://github.com/nexB/scancode.io/issues/1044

- Generate CycloneDX SBOM in 1.5 spec format, migrated from 1.4 previously.
The Package vulnerabilities are now included in the CycloneDX SBOM when available.
https://github.com/nexB/scancode.io/issues/807

- Improve the inspect_manifest pipeline to accept archives as inputs.
https://github.com/nexB/scancode.io/issues/1034

- Add support for "tagging" download URL inputs using the "<fragment>" section of URLs.
This feature is particularly useful in the map_develop_to_deploy pipeline when
download URLs are utilized as inputs. Tags such as "from" and "to" can be specified
by adding "from" or "to" fragments at the end of the download URLs.
Using the CLI, the uploaded files can be tagged using the "filename:tag" syntax
while using the `--input-file` arguments.
In the UI, tags can be edited from the Project details view "Inputs" panel.
On the REST API, a new `upload_file_tag` field is available to use along the
`upload_file`.
https://github.com/nexB/scancode.io/issues/708

--------------------

- Upgrade Django to version 5.0 and drop support for Python 3.8 and 3.9
https://github.com/nexB/scancode.io/issues/1020

- Fetching "Download URL" inputs is now delegated to an initial pipeline step that is
always run as the start of a pipeline.
This allows to run pipelines on workers running from a remote location, external to
the main ScanCode.io app server.
https://github.com/nexB/scancode.io/issues/410

- Migrate the Project.input_sources field into a InputSource model.
https://github.com/nexB/scancode.io/issues/410

- Refactor run_scancode to not fail on scan errors happening at the resource level,
such as a timeout. Project error message are created instead.
https://github.com/nexB/scancode.io/issues/1018

- Add support for the SCANCODEIO_SCAN_FILE_TIMEOUT setting in the scan_package pipeline.
https://github.com/nexB/scancode.io/issues/1018

- Add support for non-archive single file in the scan_package pipeline.
https://github.com/nexB/scancode.io/issues/1009

- Do not include "add-on" pipelines in the "New project" form choices.
https://github.com/nexB/scancode.io/issues/1041

- Display a "Run pipelines" button in the "Pipelines" panel.
Remove the ability to run a single pipeline in favor of running all "not started"
project pipeline.
https://github.com/nexB/scancode.io/issues/997

- In "map_deploy_to_develop" pipeline, add support for path patterns
in About file attributes documenting resource paths.
https://github.com/nexB/scancode.io/issues/1004

- Fix an issue where the pipeline details cannot be fetched when using URLs that
include credentials such as "user:passdomain".
https://github.com/nexB/scancode.io/issues/998

- Add a new pipeline, ``match_to_purldb``, that check CodebaseResources of a
Project against PurlDB for Package matches.

--------------------

- Display the ``Run.scancodeio_version`` in the Pipeline run modal.
When possible this value is displayed as a link to the diff view between the current
ScanCode.io version and the version used when the Pipeline was run.
https://github.com/nexB/scancode.io/issues/956

- Improve presentation of the "Resources detected license expressions" project section.
https://github.com/nexB/scancode.io/issues/937

- Add ability to sort by Package URL in package list
https://github.com/nexB/scancode.io/issues/938

- Fix an issue where the empty project settings were overriding the settings loaded
from a config file.
https://github.com/nexB/scancode.io/issues/961

- Control the execution order of Pipelines within a Project. Pipelines are not allowed
to start anymore unless all the previous ones within a Project have completed.
https://github.com/nexB/scancode.io/issues/901

- Add support for webhook subscriptions in project clone.
https://github.com/nexB/scancode.io/pull/910

- Add resources license expression summary panel in the project details view.
This panel displays the list of licenses detected in the project and include links
to the resources list.
https://github.com/nexB/scancode.io/pull/355

- Add the ``tag`` field on the DiscoveredPackage model. This new field is used to store
the layer id where the package was found in the Docker context.
https://github.com/nexB/scancode.io/issues/919

- Add to apply actions, such as archive, delete, and reset to a selection of project
from the main list.
https://github.com/nexB/scancode.io/issues/488

- Add new "Outputs" panel in the Project details view.
Output files are listed and can be downloaded from the panel.
https://github.com/nexB/scancode.io/issues/678

- Add a step in the ``deploy_to_develop`` pipelines to create "local-files" packages
with from-side resource files that have one or more relations with to-side resources
that are not part of a package.
This allows to include those files in the SBOMs and attribution outputs.
https://github.com/nexB/scancode.io/issues/914

- Enable sorting the packages list by resources count.
https://github.com/nexB/scancode.io/issues/978

--------------------

- Improve the performance of the codebase relations list view to support large number
of entries.
https://github.com/nexB/scancode.io/issues/858

- Improve DiscoveredPackageListView query performances refining the prefetch_related.
https://github.com/nexB/scancode.io/issues/856

- Fix the ``map_java_to_class`` d2d pipe to skip if no ``.java`` file is found.
https://github.com/nexB/scancode.io/issues/853

- Enhance Package search to handle full ``pkg:`` purls and segment of purls.
https://github.com/nexB/scancode.io/issues/859

- Add a new step in the ``deploy_to_develop`` pipeline where we tag archives as
processed, if all the resources in their extracted directory is mapped/processed.
https://github.com/nexB/scancode.io/issues/827

- Add the ability to clone a project.
https://github.com/nexB/scancode.io/issues/874

- Improve perceived display performance of projects charts and stats on home page.
The charts are displayed when the number of resources or packages are less than
5000 records. Else, a button to load the charts is displayed.
https://github.com/nexB/scancode.io/issues/844

- Add advanced search query system to all list views.
Refer to the documentation for details about the search syntax.
https://github.com/nexB/scancode.io/issues/871

- Migrate the ProjectError model to a global ProjectMessage.
3 level of severity available: INFO, WARNING, and ERROR.
https://github.com/nexB/scancode.io/issues/338

- Add label/tag system that can be used to group and filters projects.
https://github.com/nexB/scancode.io/issues/769

--------------------

Security release: This release addresses the security issue detailed below.
We encourage all users of ScanCode.io to upgrade as soon as possible.

- GHSA-6xcx-gx7r-rccj: Reflected Cross-Site Scripting (XSS) in license endpoint
The ``license_details_view`` function was subject to cross-site scripting (XSS)
attack due to inadequate validation and sanitization of the key parameter.
The license views were migrated class-based views are the inputs are now properly
sanitized.
Credit to 0xmpij for reporting the vulnerability.
https://github.com/nexB/scancode.io/security/advisories/GHSA-6xcx-gx7r-rccj
https://github.com/nexB/scancode.io/issues/847

- Add bandit analyzer and Django "check --deploy" to the check/validation stack.
This helps to ensure that we do not introduce know code vulnerabilities and
deployment issues to the codebase.
https://github.com/nexB/scancode.io/issues/850

- Migrate the run_command function into a safer usage of the subprocess module.
Also fix various warnings returned by the bandit analyzer.
https://github.com/nexB/scancode.io/issues/850

- Replace the ``scancode.run_scancode`` function by a new ``run_scan`` that interact
with scancode-toolkit scanners without using subprocess. This new function is used
in the ``scan_package`` pipeline.
The ``SCANCODE_TOOLKIT_CLI_OPTIONS`` settings was renamed
``SCANCODE_TOOLKIT_RUN_SCAN_ARGS``. Refer to the documentation for the next "dict"
syntax.
https://github.com/nexB/scancode.io/issues/798

--------------------

Security release: This release addresses the security issue detailed below.
We encourage all users of ScanCode.io to upgrade as soon as possible.

- GHSA-2ggp-cmvm-f62f: Command injection in docker image fetch process
The ``fetch_docker_image`` function was subject to potential injection attack.
The user inputs are now sanitized before calling the subprocess function.
Credit to 0xmpij for reporting the vulnerability.
https://github.com/nexB/scancode.io/security/advisories/GHSA-2ggp-cmvm-f62f

---

- Add support for multiple input URLs, and adding multiple pipelines in the project
creation REST API.
https://github.com/nexB/scancode.io/issues/828

- Update the ``fetch_vulnerabilities`` pipe to make the API requests by batch of purls.
https://github.com/nexB/scancode.io/issues/835

- Add vulnerability support for discovered dependencies.
The dependency data is loaded using the ``find_vulnerabilities`` pipeline backed by
a VulnerableCode database.
https://github.com/nexB/scancode.io/issues/835

- Fix root filesystem scanning for installed packages and archived Linux distributions.
Allows the scan to discover system packages from `rpmdb.sqlite` and other sources.
https://github.com/nexB/scancode.io/pull/840