Scancodeio

--------------------

- Display the ``Run.scancodeio_version`` in the Pipeline run modal.
When possible this value is displayed as a link to the diff view between the current
ScanCode.io version and the version used when the Pipeline was run.
https://github.com/nexB/scancode.io/issues/956

- Improve presentation of the "Resources detected license expressions" project section.
https://github.com/nexB/scancode.io/issues/937

- Add ability to sort by Package URL in package list
https://github.com/nexB/scancode.io/issues/938

- Fix an issue where the empty project settings were overriding the settings loaded
from a config file.
https://github.com/nexB/scancode.io/issues/961

- Control the execution order of Pipelines within a Project. Pipelines are not allowed
to start anymore unless all the previous ones within a Project have completed.
https://github.com/nexB/scancode.io/issues/901

- Add support for webhook subscriptions in project clone.
https://github.com/nexB/scancode.io/pull/910

- Add resources license expression summary panel in the project details view.
This panel displays the list of licenses detected in the project and include links
to the resources list.
https://github.com/nexB/scancode.io/pull/355

- Add the ``tag`` field on the DiscoveredPackage model. This new field is used to store
the layer id where the package was found in the Docker context.
https://github.com/nexB/scancode.io/issues/919

- Add to apply actions, such as archive, delete, and reset to a selection of project
from the main list.
https://github.com/nexB/scancode.io/issues/488

- Add new "Outputs" panel in the Project details view.
Output files are listed and can be downloaded from the panel.
https://github.com/nexB/scancode.io/issues/678

- Add a step in the ``deploy_to_develop`` pipelines to create "local-files" packages
with from-side resource files that have one or more relations with to-side resources
that are not part of a package.
This allows to include those files in the SBOMs and attribution outputs.
https://github.com/nexB/scancode.io/issues/914

- Enable sorting the packages list by resources count.
https://github.com/nexB/scancode.io/issues/978

--------------------

- Improve the performance of the codebase relations list view to support large number
of entries.
https://github.com/nexB/scancode.io/issues/858

- Improve DiscoveredPackageListView query performances refining the prefetch_related.
https://github.com/nexB/scancode.io/issues/856

- Fix the ``map_java_to_class`` d2d pipe to skip if no ``.java`` file is found.
https://github.com/nexB/scancode.io/issues/853

- Enhance Package search to handle full ``pkg:`` purls and segment of purls.
https://github.com/nexB/scancode.io/issues/859

- Add a new step in the ``deploy_to_develop`` pipeline where we tag archives as
processed, if all the resources in their extracted directory is mapped/processed.
https://github.com/nexB/scancode.io/issues/827

- Add the ability to clone a project.
https://github.com/nexB/scancode.io/issues/874

- Improve perceived display performance of projects charts and stats on home page.
The charts are displayed when the number of resources or packages are less than
5000 records. Else, a button to load the charts is displayed.
https://github.com/nexB/scancode.io/issues/844

- Add advanced search query system to all list views.
Refer to the documentation for details about the search syntax.
https://github.com/nexB/scancode.io/issues/871

- Migrate the ProjectError model to a global ProjectMessage.
3 level of severity available: INFO, WARNING, and ERROR.
https://github.com/nexB/scancode.io/issues/338

- Add label/tag system that can be used to group and filters projects.
https://github.com/nexB/scancode.io/issues/769

--------------------

Security release: This release addresses the security issue detailed below.
We encourage all users of ScanCode.io to upgrade as soon as possible.

- GHSA-6xcx-gx7r-rccj: Reflected Cross-Site Scripting (XSS) in license endpoint
The ``license_details_view`` function was subject to cross-site scripting (XSS)
attack due to inadequate validation and sanitization of the key parameter.
The license views were migrated class-based views are the inputs are now properly
sanitized.
Credit to 0xmpij for reporting the vulnerability.
https://github.com/nexB/scancode.io/security/advisories/GHSA-6xcx-gx7r-rccj
https://github.com/nexB/scancode.io/issues/847

- Add bandit analyzer and Django "check --deploy" to the check/validation stack.
This helps to ensure that we do not introduce know code vulnerabilities and
deployment issues to the codebase.
https://github.com/nexB/scancode.io/issues/850

- Migrate the run_command function into a safer usage of the subprocess module.
Also fix various warnings returned by the bandit analyzer.
https://github.com/nexB/scancode.io/issues/850

- Replace the ``scancode.run_scancode`` function by a new ``run_scan`` that interact
with scancode-toolkit scanners without using subprocess. This new function is used
in the ``scan_package`` pipeline.
The ``SCANCODE_TOOLKIT_CLI_OPTIONS`` settings was renamed
``SCANCODE_TOOLKIT_RUN_SCAN_ARGS``. Refer to the documentation for the next "dict"
syntax.
https://github.com/nexB/scancode.io/issues/798

--------------------

Security release: This release addresses the security issue detailed below.
We encourage all users of ScanCode.io to upgrade as soon as possible.

- GHSA-2ggp-cmvm-f62f: Command injection in docker image fetch process
The ``fetch_docker_image`` function was subject to potential injection attack.
The user inputs are now sanitized before calling the subprocess function.
Credit to 0xmpij for reporting the vulnerability.
https://github.com/nexB/scancode.io/security/advisories/GHSA-2ggp-cmvm-f62f

---

- Add support for multiple input URLs, and adding multiple pipelines in the project
creation REST API.
https://github.com/nexB/scancode.io/issues/828

- Update the ``fetch_vulnerabilities`` pipe to make the API requests by batch of purls.
https://github.com/nexB/scancode.io/issues/835

- Add vulnerability support for discovered dependencies.
The dependency data is loaded using the ``find_vulnerabilities`` pipeline backed by
a VulnerableCode database.
https://github.com/nexB/scancode.io/issues/835

- Fix root filesystem scanning for installed packages and archived Linux distributions.
Allows the scan to discover system packages from `rpmdb.sqlite` and other sources.
https://github.com/nexB/scancode.io/pull/840

--------------------

WARNING: After upgrading the ScanCode.io codebase to this version,
and following the ``docker compose build``,
the permissions of the ``/var/scancodeio/`` directory of the Docker volumes require
to be updated for the new ``app`` user, using:
``docker compose run -u 0:0 web chown -R app:app /var/scancodeio/``

- Run Docker as non-root user using virtualenv.
WARNING: The permissions of the ``/var/scancodeio/`` directory in the Docker volumes
require to be updated for the new ``app`` user.
https://github.com/nexB/scancode.io/issues/399

- Add column sort and filters in dependency list view.
https://github.com/nexB/scancode.io/issues/823

- Add a new ``ScanCodebasePackage`` pipeline to scan a codebase for packages only.
https://github.com/nexB/scancode.io/issues/815

- Add new ``outputs`` REST API action that list projects output files including an URL
to download the file.
https://github.com/nexB/scancode.io/issues/678

- Add support for multiple to/from input files in the ``deploy_to_develop`` pipeline.
https://github.com/nexB/scancode.io/issues/813

- Add the ability to delete and download project inputs.
Note that the inputs cannot be modified (added or deleted) once a pipeline run as
started on the project.
https://github.com/nexB/scancode.io/issues/813

- Fix root_filesystem data structure stored on the Project ``extra_data`` field.
This was causing a conflict with the expected docker images data structure
when generating an XLSX output.
https://github.com/nexB/scancode.io/issues/824

- Fix the SPDX output to include missing detailed license texts for LicenseRef.
Add ``licensedb_url`` and ``scancode_url`` to the SPDX ``ExtractedLicensingInfo``
``seeAlsos``.
Include the ``Package.notice_text`` as the SPDX ``attribution_texts``.
https://github.com/nexB/scancode.io/issues/841

--------------------

- Add support for license policies and complaince alert for Discovered Packages.
https://github.com/nexB/scancode.io/issues/151

- Refine the details views and tabs:
- Add a "Relations" tab in the Resource details view
- Disable empty tabs by default
- Display the count of items in the tab label
- Improve query performances for details views
https://github.com/nexB/scancode.io/issues/799

- Upgrade vulnerablecode integration:
- Add ``affected_by_vulnerabilities`` field on ``DiscoveredPackage`` model.
- Add UI for showing package vulnerabilities in details view.
- Add packages filtering by ``is_vulnerable``.
- Include vulnerability data in the JSON results.
https://github.com/nexB/scancode.io/issues/600

- Add multiple new filtering option to list views table headers.
Refactored the way to define filters using the table_columns view attribute.
https://github.com/nexB/scancode.io/issues/216
https://github.com/nexB/scancode.io/issues/580
https://github.com/nexB/scancode.io/issues/506

- Update the CycloneDX BOM download file extension from ``.bom.json`` to ``.cdx.json``.
https://github.com/nexB/scancode.io/issues/785

- SPDX download BOM do not include codebase resource files by default anymore.
https://github.com/nexB/scancode.io/issues/785

- Add archive_location to the LAYERS worksheet of XLSX output.
https://github.com/nexB/scancode.io/issues/773

- Add "New Project" button to Project details view.
https://github.com/nexB/scancode.io/issues/763

- Display image type files in the codebase resource details view in a new "Image" tab.

- Add ``slug`` field on the Project model. That field is used in URLs instead of the
``uuid``.
https://github.com/nexB/scancode.io/issues/745

- Fix the ordering of the Codebase panel in the Project details view.
https://github.com/nexB/scancode.io/issues/795

- Do not rely on the internal ``id`` PK for package and dependency details URLs.
Package details URL is now based on ``uuid`` and the dependency details URL is based
on ``dependency_uid``.
https://github.com/nexB/scancode.io/issues/331

- Add a "License score" project setting that can be used to limit the returned license
matches with a score above the provided one.
This is leveraging the ScanCode-toolkit ``--license-score`` option, see:
https://scancode-toolkit.readthedocs.io/en/stable/cli-reference/basic-options.html#license-score-option
https://github.com/nexB/scancode.io/issues/335