**Feature removals:**
* Drop Python 3.6 support\
Python 3.6 has reached the end of life in Dec 2021. This means it
won't receive any further patches to fix the secutiry issues. So,
dropping the support of 3.6 and it now supports Python>=3.7 version
only.
**New components:**
* grimoirelab-toolkit 0.3.0
* perceval 0.18.0
* perceval-mozilla 0.3.0
* perceval-opnfv 0.2.0
* perceval-puppet 0.2.0
* perceval-weblate 0.2.0
* graal 0.3.0
* kingarthur 0.2.0
* sortinghat 0.7.20
* cereslib 0.3.0
* grimoire-elk 0.101.1
* sigils 0.1.0
* sirmordred 0.3.0
* kidash 0.5.0
The following list describes the changes by component:
Perceval
**New features:**
* [bugzillarest] API Key authentication\
A new authentication is available in the `bugzillarest` backend using
an API Key. This can be provided using the parameter `--api-key` on
the command line. Note that this parameter will invalidate `--backend-
user` `--backend-password`, and `--api-token` parameters. For
developers, this parameter is also available during the initialization
of the class `BugzillaRESTClient` under the name `api_key`.
* Confluence supports fetch by spaces\
This allows fetching confluence by specific spaces. By default it will
fetch the entire instance, but adding the `--spaces` argument will
fetch only these spaces.
Graal
**Bug fixes:**
* Fix error in update worktree\
Graal wasn't working with the latest version of Git 2.35.1 because it
protects braches checked out in all worktrees. This change renames the
branch created when creating a worktree to fix the issue.
Sorting Hat
**Bug fixes:**
* [gitdm] Skip invalid format lines\
Gitdm parser won't fail reading files with an invalid format. Instead,
it will ignore invalid content.
GrimoireELK
**New features:**
* Anonymize parameters\
A new API function `anonymize_params(params)` allows to anonymize a
list of parameters that developers might consider secrets. It will
only take effect for those parameters which their names are defined on
`SECRET_PARAMETERS` constant.
* Dependencies file for development\
Include a new requirements.txt file to install the development version
from the repositories.
* Support to add identities to SortingHat from Enrich class\
New methods `add_identities` and `add_identity` are available on
`Enrich` class to add new identities to a `SortingHat` database.
Libraries using `elk` won't need to use `sortinghat` library to store
new identities while enriching items.
* Collect and enrich confluence by spaces\
Users can collect and enrich Confluence data from specific spaces
only, adding the option `--spaces=[SPACE1, SPACE2, ...]` to their
`projects.json`. The following example shows how to collect and to
enrich data from only the `DEV` and `SUPP` spaces only. {
"project": { "confluence": [ "http://example.com
--spaces=[DEV, SUPP]" ] } }
* Add demographic study to mattermost\
Add the demographic study to the mattermost data source.
**Bug fixes:**
* Skip sources with wrong arguments\
Sources won't interrupt collection or enrichment processes when any of
their arguments are invalid. Instead, they will be skipped.
* Git fix 'is_git_commit_multi_author'\
This fixes the `is_git_commit_multi_author` field when the commit has
more than one author and `pair-programming` is enabled. This is
important for calculating `pair_programming_*` values.
* SortingHat as a required package\
SortingHat is required for running grimoireelk but it was set as an
extra dependency. From now on sortinghat is always installed with
grimoireelk.
* Confluence credentials not stored in raw indexes\
Credentials for Confluence datasource were stored in raw indices as
part of the URL. For now on, credentials that are part of URLs are
removed before any data is stored.
* Pair programming regular expression improvement\
The regular expression to detect pair programming authors in git
datasource has been improved adding more connectors. The following
list shows the current connectors: - `[aA][nN][dD]` - `&` - `+`
* Fix fetching authors with min and max dates\
The old query only returns 10000 items due to ElasticSearch and
OpenSearch now use `composite` aggregation to paginate all buckets.
Sir Mordred
**New features:**
* Add extract spaces\
Extract spaces from the URL. By default it will extract labels but
adding `tag_type="spaces"` it will extract spaces.
**Bug fixes:**
* Studies aliases relocated\
Relocate studies aliases so they are not created automatically by
Mordred.
* Add pair-programming in '_get_enrich_backend' method\
Add the `pair-programming` to the `_get_enrich_backend` method if it
exists in the setup.cfg file.
**Feature removals:**
* FINOS Meeting backend removed\
FINOS Meeting backend is not supported anymore. The Perceval backend
was archived in Bitergia repository so it shouldn't be part of the
core anymore because it's software not maintained.
**Others:**
* Add studies to the alias file\
Add onion, areas of code, and demographics studies to the file
aliases.json. This is needed to keep this file updated with the
aliases we use in the dashboards by adding them to the appropiate data
sources. The included aliases: - all_onion - git_areas_of_code -
demographics
Releases
cereslib 1.0.8 - (2024-12-11)
* Update Poetry's package dependencies
cereslib 1.0.7 - (2024-11-13)
* Update Poetry's package dependencies
cereslib 1.0.6 - (2024-10-15)
* Update Poetry's package dependencies
cereslib 1.0.5 - (2024-09-23)
**Dependencies updateds:**
* Python minimum version updated\
Python 3.8 will reach its end of life in October 2024. Python 3.9 is
the minimum version required by the project.
cereslib 1.0.4 - (2024-08-09)
* Update Poetry's package dependencies
cereslib 1.0.3 - (2024-08-02)
* Update Poetry's package dependencies
cereslib 1.0.2 - (2024-06-21)
* Update Poetry's package dependencies
cereslib 1.0.1 - (2024-05-09)
* Update Poetry's package dependencies
cereslib 1.0.0 - (2024-04-13)
**New features:**
* First major release\
GrimoireLab reached a stable status. This is our first major release.
cereslib 0.5.6 - (2024-03-27)
* Update Poetry's package dependencies
cereslib 0.5.5 - (2024-03-12)
* Update Poetry's package dependencies
cereslib 0.5.4 - (2024-03-01)
* Update Poetry's package dependencies
cereslib 0.5.3 - (2024-02-08)
* Update Poetry's package dependencies
cereslib 0.5.2 - (2024-01-30)
* Update Poetry's package dependencies
cereslib 0.5.1 - (2023-12-19)
* Update Poetry's package dependencies
cereslib 0.5.0 - (2023-11-28)
**New features:**
* TimeZone field in Git Areas of Code\
Include time zone field of commit date in Git Areas of Code index.
**Bug fixes:**
* Future compatibility with Pandas method (52)\
Set regex parameter to True in `pandas.Series.str.replace` for future
compatibility.
cereslib 0.4.3 - (2023-10-20)
**Dependencies updateds:**
* Add Python 3.9 and 3.10 and drop 3.7 support\
Python 3.7 reached the end of life phase on June 27 2023 and is no
longer supported.
cereslib 0.4.2 - (2023-07-11)
* Update Poetry's package dependencies
cereslib 0.4.1 - (2023-06-28)
* Update Poetry's package dependencies
cereslib 0.4.0 - (2023-04-21)
**New features:**
* Support for more languages and file types\
The library is able to detect the new file formats and language
formats. This list includes JavaScript, TypeScript, Terraform, among
others.
cereslib 0.3.6 - (2023-02-01)
* Update Poetry's package dependencies
cereslib 0.3.5 - (2022-11-07)
* Update Poetry's package dependencies
cereslib 0.3.4 - (2022-10-31)
* Update Poetry's package dependencies
cereslib 0.3.2 - (2022-09-26)
**Others:**
* Update package dependencies\
Update numpy version to be the same as in sortinghat
cereslib 0.3.2-rc.4 - (2022-09-26)
**Others:**
* Update package dependencies\
Update numpy version to be the same as in sortinghat
cereslib 0.3.2-rc.3 - (2022-09-26)
**Others:**
* Update package dependencies\
Update numpy version to be the same as in sortinghat
cereslib 0.3.1 - (2022-06-24)
**Others:**
* GrimoireLab dependencies update\
Dependencies have been updated to use the latest versions of
GrimoireLab packages:
- grimoirelab-toolkit >= 0.3
cereslib 0.3.0 - (2022-06-03)
**New features:**
* Dependencies file for development\
Include a new requirements.txt file to install the development version
from the repositories.
cereslib 0.2.0 - (2022-05-31)
**Feature removals:**
* grimoire-elk dependency removed\
The dependency of the library 'grimoire-elk' has been removed to avoid
the cyclic dependency that has with 'cereslib'.
Releases
kingarthur 0.2.5 - (2022-11-07)
* Update Poetry's package dependencies
kingarthur 0.2.4 - (2022-10-31)
* Update Poetry's package dependencies
King Arthur 0.2.1 - (2022-06-24)
**Others:**
* GrimoireLab dependencies update\
Dependencies have been updated to use the latest versions of
GrimoireLab packages:
- grimoirelab-toolkit >= 0.3
- perceval >= 0.19
King Arthur 0.2.0 - (2022-06-03)
**New features:**
* Dependencies file for development\
Include a new requirements.txt file to install the development version
from the repositories.
**Feature removals:**
* Drop Python 3.6 support\
Python 3.6 reached the end of life at the end of 2021. This means it
won't receive new updates or patches to fix security issues.
Therefore, this package will only work with Python >= 3.7 from now on.
Releases
perceval-puppet 1.0.10 - (2024-12-11)
* Update Poetry's package dependencies
perceval-puppet 1.0.9 - (2024-11-13)
* Update Poetry's package dependencies
perceval-puppet 1.0.8 - (2024-10-15)
* Update Poetry's package dependencies
perceval-puppet 1.0.7 - (2024-09-23)
**Dependencies updateds:**
* Python minimum version updated\
Python 3.8 will reach its end of life in October 2024. Python 3.9 is
the minimum version required by the project.
perceval-puppet 1.0.6 - (2024-08-30)
* Update Poetry's package dependencies
perceval-puppet 1.0.5 - (2024-08-13)
* Update Poetry's package dependencies
perceval-puppet 1.0.4 - (2024-08-09)
* Update Poetry's package dependencies
perceval-puppet 1.0.3 - (2024-08-02)
* Update Poetry's package dependencies
perceval-puppet 1.0.2 - (2024-06-21)
* Update Poetry's package dependencies
perceval-puppet 1.0.1 - (2024-05-09)
* Update Poetry's package dependencies
perceval-puppet 1.0.0 - (2024-04-13)
**New features:**
* First major release\
GrimoireLab reached a stable status. This is our first major release.
perceval-puppet 0.2.25 - (2024-03-27)
* Update Poetry's package dependencies
perceval-puppet 0.2.24 - (2024-03-12)
* Update Poetry's package dependencies
perceval-puppet 0.2.23 - (2024-03-01)
* Update Poetry's package dependencies
perceval-puppet 0.2.22 - (2024-02-08)
* Update Poetry's package dependencies
perceval-puppet 0.2.21 - (2024-01-30)
* Update Poetry's package dependencies
perceval-puppet 0.2.20 - (2023-12-19)
* Update Poetry's package dependencies
perceval-puppet 0.2.19 - (2023-11-28)
* Update Poetry's package dependencies
perceval-puppet 0.2.18 - (2023-11-14)
* Update Poetry's package dependencies
perceval-puppet 0.2.17 - (2023-11-03)
* Update Poetry's package dependencies
perceval-puppet 0.2.16 - (2023-10-20)
**Dependencies updateds:**
* Add Python 3.10 and 3.11 and drop 3.7 support\
Python 3.7 reached the end of life phase on June 27 2023 and is no
longer supported.
perceval-puppet 0.2.15 - (2023-08-06)
* Update Poetry's package dependencies
perceval-puppet 0.2.14 - (2023-07-23)
* Update Poetry's package dependencies
perceval-puppet 0.2.13 - (2023-07-11)
* Update Poetry's package dependencies
perceval-puppet 0.2.12 - (2023-06-28)
* Update Poetry's package dependencies
perceval-puppet 0.2.11 - (2023-05-17)
* Update Poetry's package dependencies
perceval-puppet 0.2.10 - (2023-04-27)
* Update Poetry's package dependencies
perceval-puppet 0.2.9 - (2023-04-26)
* Update Poetry's package dependencies
perceval-puppet 0.2.8 - (2023-04-21)
* Update Poetry's package dependencies
perceval-puppet 0.2.7 - (2023-02-03)
* Update Poetry's package dependencies
perceval-puppet 0.2.6 - (2023-02-01)
* Update Poetry's package dependencies
perceval-puppet 0.2.5 - (2022-11-07)
* Update Poetry's package dependencies
perceval-puppet 0.2.4 - (2022-10-31)
* Update Poetry's package dependencies
perceval-puppet 0.2.1 - (2022-06-24)
**Others:**
* GrimoireLab dependencies update\
Dependencies have been updated to use the latest versions of
GrimoireLab packages:
- grimoirelab-toolkit >= 0.3
- perceval >= 0.19
perceval-puppet 0.2.0 - (2022-06-02)
**New features:**
* Dependencies file for development\
Include a new requirements.txt file to install the development version
from the repositories.
Releases
grimoirelab-toolkit 1.0.8 - (2024-12-11)
* Update Poetry's package dependencies
grimoirelab-toolkit 1.0.7 - (2024-11-13)
* Update Poetry's package dependencies
grimoirelab-toolkit 1.0.6 - (2024-10-15)
* Update Poetry's package dependencies
grimoirelab-toolkit 1.0.5 - (2024-09-23)
**Dependencies updateds:**
* Python minimum version updated\
Python 3.8 will reach its end of life in October 2024. Python 3.9 is
the minimum version required by the project.
grimoirelab-toolkit 1.0.4 - (2024-08-09)
* Update Poetry's package dependencies
grimoirelab-toolkit 1.0.3 - (2024-08-02)
* Update Poetry's package dependencies
grimoirelab-toolkit 1.0.2 - (2024-06-21)
* Update Poetry's package dependencies
grimoirelab-toolkit 1.0.1 - (2024-05-09)
* Update Poetry's package dependencies
grimoirelab-toolkit 1.0.0 - (2024-04-13)
**New features:**
* First major release\
GrimoireLab reached a stable status. This is our first major release.
grimoirelab-toolkit 1.0.0-rc.1 - (2024-04-09)
**New features:**
* First major release\
GrimoireLab reached a stable status. This is our first major release.
grimoirelab-toolkit 0.4.5 - (2024-03-27)
* Update Poetry's package dependencies
grimoirelab-toolkit 0.4.4 - (2024-03-12)
* Update Poetry's package dependencies
grimoirelab-toolkit 0.4.3 - (2024-03-01)
* Update Poetry's package dependencies
grimoirelab-toolkit 0.4.2 - (2024-01-30)
* Update Poetry's package dependencies
grimoirelab-toolkit 0.4.1 - (2023-12-19)
* Update Poetry's package dependencies
grimoirelab-toolkit 0.4.0 - (2023-10-20)
**Feature removals:**
* Drop Python 3.7 support\
Python 3.7 reaches the end of life in June 2023. This means it won't
receive new updates or patches to fix security issues. Therefore, this
package only works with Python >= 3.8 from now on.
grimoirelab-toolkit 0.3.6 - (2023-07-11)
* Update Poetry's package dependencies
grimoirelab-toolkit 0.3.5 - (2023-06-28)
* Update Poetry's package dependencies
grimoirelab-toolkit 0.3.4 - (2023-04-21)
* Update Poetry's package dependencies
grimoirelab-toolkit 0.3.3 - (2023-02-01)
* Update Poetry's package dependencies
grimoirelab-toolkit 0.3.2 - (2022-10-31)
* Update Poetry's package dependencies
grimoirelab-toolkit 0.3.0 - (2022-05-31)
**Feature removals:**
* Drop Python 3.6 support\
Python 3.6 has reached the end of life in Dec 2021. This means it
won't receive any further patches to fix the secutiry issues. So,
dropping the support of 3.6 and it now supports Python>=3.7 version
only.
grimoirelab-toolkit 0.2.0 - (2021-04-20)
**Security fixes:**
* Drop Python < 3.6 support \
Python versions previous to 3.6 ended their security support life in
September 2020. This new release drops its support too.
grimoirelab-toolkit 0.1.0 - (2017-05-15)
** New features and improvements: **
* This first release contains common functions used across other
GrimoireLab projects. These functions handle datetime objects,
URIs/URLs and high-level introspection tools.
Releases
kidash 1.0.9 - (2024-12-11)
* Update Poetry's package dependencies
kidash 1.0.8 - (2024-11-13)
* Update Poetry's package dependencies
kidash 1.0.7 - (2024-10-15)
* Update Poetry's package dependencies
kidash 1.0.6 - (2024-09-23)
**Dependencies updateds:**
* Python minimum version updated\
Python 3.8 will reach its end of life in October 2024. Python 3.9 is
the minimum version required by the project.
kidash 1.0.5 - (2024-08-30)
* Update Poetry's package dependencies
kidash 1.0.4 - (2024-08-09)
* Update Poetry's package dependencies
kidash 1.0.3 - (2024-08-02)
* Update Poetry's package dependencies
kidash 1.0.2 - (2024-06-21)
* Update Poetry's package dependencies
kidash 1.0.1 - (2024-05-09)
* Update Poetry's package dependencies
kidash 1.0.0 - (2024-04-13)
**New features:**
* First major release\
GrimoireLab reached a stable status. This is our first major release.
kidash 0.6.9 - (2024-03-27)
* Update Poetry's package dependencies
kidash 0.6.8 - (2024-03-12)
* Update Poetry's package dependencies
kidash 0.6.7 - (2024-03-01)
* Update Poetry's package dependencies
kidash 0.6.6 - (2024-02-08)
* Update Poetry's package dependencies
kidash 0.6.5 - (2024-01-30)
* Update Poetry's package dependencies
kidash 0.6.4 - (2023-12-19)
**Dependencies updateds:**
* Downgrade urllib3 from ^2.0.7 to ^1.26\
Downgrade the version of urllib3 to match Perceval version.
kidash 0.6.3 - (2023-11-28)
* Update Poetry's package dependencies
kidash 0.6.2 - (2023-11-14)
* Update Poetry's package dependencies
kidash 0.6.1 - (2023-11-03)
* Update Poetry's package dependencies
kidash 0.6.0 - (2023-10-20)
**Feature removals:**
* Drop Python 3.7 support\
Python 3.7 reaches the end of life in June 2023. This means it won't
receive new updates or patches to fix security issues. Therefore, this
package only works with Python >= 3.8 from now on.
**Dependencies updateds:**
* Bump urllib3 version\
Bump urllib3 from ^1.26 to ^2.0.7
kidash 0.5.9 - (2023-07-23)
* Update Poetry's package dependencies
kidash 0.5.8 - (2023-07-11)
* Update Poetry's package dependencies
kidash 0.5.7 - (2023-06-28)
* Update Poetry's package dependencies
kidash 0.5.6 - (2023-05-17)
* Update Poetry's package dependencies
kidash 0.5.5 - (2023-04-27)
* Update Poetry's package dependencies
kidash 0.5.4 - (2023-04-21)
* Update Poetry's package dependencies
kidash 0.5.3 - (2023-02-01)
* Update Poetry's package dependencies
kidash 0.5.2 - (2022-10-31)
* Update Poetry's package dependencies
kidash 0.5.1 - (2022-09-26)
**Others:**
* Update Poetry dependencies\
Update package dependencies for the new release.
kidash 0.5.1-rc.7 - (2022-09-26)
**Others:**
* Update Poetry dependencies\
Update package dependencies for the new release.
kidash 0.5.1-rc.6 - (2022-09-26)
**Others:**
* Update Poetry dependencies\
Update package dependencies for the new release.
kidash 0.5.1-rc.5 - (2022-09-26)
**Others:**
* Update Poetry dependencies\
Update package dependencies for the new release.
kidash 0.5.1-rc.4 - (2022-09-23)
**Others:**
* Update Poetry dependencies\
Update package dependencies for the new release.
kidash 0.5.0 - (2022-06-02)
**Feature removals:**
* Drop Python 3.6 support\
Python 3.6 reached the end of life at the end of 2021. This means it
won't receive new updates or patches to fix security issues.
Therefore, this package will only work with Python >= 3.7 from now on.
Releases
perceval 1.1.0 - (2024-12-11)
**Bug fixes:**
* Git stats not assigned with their actions\
Git stats were not assigned to their actions in a commit for filenames
that contain whitespaces. Instead, an empty stat was created like in a
merge commit, where actions normally don't take place.
**Performance improvements:**
* Git sync improved for `--latest-items` flag\
The speed updating a git repository when `--lates-items` is set was
very poor. The main problem was when the references were updated to
the new hashes. All the active refs on the repository were modified,
even when they were up-to-date. Now, only references updated on the
remote repository are modified on the local one. Repositories are
synched 2 times faster than before.
perceval 1.0.9 - (2024-11-13)
* Update Poetry's package dependencies
perceval 1.0.8 - (2024-10-15)
* Update Poetry's package dependencies
perceval 1.0.7 - (2024-09-23)
**Dependencies updateds:**
* Python minimum version updated\
Python 3.8 will reach its end of life in October 2024. Python 3.9 is
the minimum version required by the project.
perceval 1.0.6 - (2024-08-30)
**Bug fixes:**
* Fix issue 782 (782)\
The issue was that perceval would not delete old tags from upstream
references. This change deletes tags locally if tags are deleted
upstream.
perceval 1.0.5 - (2024-08-13)
**Bug fixes:**
* Fixed missing backend name in usage string (799)\
The `perceval` command-line tool now correctly displays the backend
name in the usage string when using the `--help` option.
perceval 1.0.4 - (2024-08-09)
* Update Poetry's package dependencies
perceval 1.0.3 - (2024-08-02)
* Update Poetry's package dependencies
perceval 1.0.2 - (2024-06-21)
**Bug fixes:**
* Empty stats for moved/copied files in git\
Stats about changes on a file were not reported correctly for files
that where moved to a subdirectory. They were reported as an invalid
entry and without action associated. For example, the file
`dir/filename` was moved to `dir/subdir/filename`, but it was reported
as `dir//filename`. Therefore, the entry of the file
`dir/subdire/filename` didn't have the stats about added and deleted
lines. This error has been fixed and stats are reported correctly.
perceval 1.0.1 - (2024-05-09)
* Update Poetry's package dependencies
perceval 1.0.0 - (2024-04-13)
**New features:**
* First major release\
GrimoireLab reached a stable status. This is our first major release.
perceval 0.25.0 - (2024-03-27)
**New features:**
* Include recovery mode for Git backend\
Include a new option in Git that allows continuing to fetch commits
from the previous execution using the last commit. The option is
`--recovery <commit>`. The last commit can be obtained from the offset
in the summary object of the last execution or the last item.
perceval 0.24.1 - (2024-03-12)
* Update Poetry's package dependencies
perceval 0.24.0 - (2024-03-01)
**New deprecations:**
* NNTP backend deprecated\
NNTP is deprecated. It is not compatible with Python 3.11.8 and will
be removed in a future version.
perceval 0.23.6 - (2024-02-08)
* Update Poetry's package dependencies
perceval 0.23.5 - (2024-01-30)
* Update Poetry's package dependencies
perceval 0.23.4 - (2023-12-19)
* Update Poetry's package dependencies
perceval 0.23.3 - (2023-11-28)
* Update Poetry's package dependencies
perceval 0.23.2 - (2023-11-14)
* Update Poetry's package dependencies
perceval 0.23.1 - (2023-11-03)
* Update Poetry's package dependencies
perceval 0.23.0 - (2023-10-20)
**New features:**
* Jira authentication with token (813)\
Authentication in `jira` backend is supported using personal access
tokens. Jira Core/Software (8.14 and later), Jira Service Management
(4.15 and later) Data Center and server editions can use personal
access tokens without a username. For Jira Cloud, username and token
are required. Usage: perceval jira <YOUR_JIRA_SERVER> -u <USERNAME> -p
<PASSWORD> perceval jira <YOUR_JIRA_SERVER> -t <PERSONAL_ACCESS_TOKEN>
perceval jira <YOUR_JIRA_CLOUD_SITE> -u <USERNAME> -t <TOKEN>
* Clonfluence authentication with personal access tokens\
Authentication in `confluence` backend is supported using personal
access tokens. Confluence Data Center and server editions 7.9 and
later can use personal access tokens without a username. For
Confluence Cloud, username and token are required.
**Feature removals:**
* Drop Python 3.7 support\
Python 3.7 reaches the end of life in June 2023. This means it won't
receive new updates or patches to fix security issues. Therefore, this
package only works with Python >= 3.8 from now on.
**Dependencies updateds:**
* Bump urllib3 version\
Bump urllib3 from ^1.26 to ^2.0.7
perceval 0.22.3 - (2023-08-06)
* Update Poetry's package dependencies
perceval 0.22.2 - (2023-07-23)
* Update Poetry's package dependencies
perceval 0.22.1 - (2023-07-11)
* Update Poetry's package dependencies
perceval 0.22.0 - (2023-06-28)
**New features:**
* Git backend alternates mechanism\
Git alternates is a mechanism that lets a repository borrow objects
from another repository on the same machine. Allow to execute Git
Perceval backend using this mechanism.
perceval 0.21.7 - (2023-05-17)
* Update Poetry's package dependencies
perceval 0.21.6 - (2023-04-27)
* Update Poetry's package dependencies
perceval 0.21.5 - (2023-04-26)
* Update Poetry's package dependencies
perceval 0.21.4 - (2023-04-21)
**Bug fixes:**
* Default value for mbox to-date (810)\
Include a default value for `--to-date` argument in mbox backend.
perceval 0.21.3 - (2023-02-03)
* Update Poetry's package dependencies
perceval 0.21.2 - (2023-02-01)
* Update Poetry's package dependencies
perceval 0.21.1 - (2022-11-07)
**Bug fixes:**
* Mbox to-date argument fixed\
It fixes the `--to-date` argument. This parameter was added to fetch
data until a specific date.
perceval 0.21.0 - (2022-10-31)
**New features:**
* [gitlab] Alias `--url` for `--enterprise-url` argument (801)\
The argument `--enterprise-url` for Gitlab backend has the new alias
`--url`. With this argument, users can define the GitLab instance from
items will be fetched. This can be set to the SaaS GitLab
(`gitlab.com`) or to a self-managed instance (e.g.
`gitlab.example.com`). By default, this parameter points to
`https://gitlab.com`.
* [hyperkitty/mbox] Option `to-date` to fetch until that date\
This parameter allows fetching data until a specific date. By default,
this value is `2100-01-01`, but this can be changed adding `--to-
date`.
perceval 0.20.0 - (2022-09-26)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.15 - (2022-09-26)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.14 - (2022-09-26)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.13 - (2022-09-26)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.12 - (2022-09-23)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.11 - (2022-09-23)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.10 - (2022-09-23)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.9 - (2022-09-09)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.8 - (2022-09-09)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.7 - (2022-09-07)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.6 - (2022-09-07)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.5 - (2022-09-06)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.4 - (2022-09-06)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.3 - (2022-08-23)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.2 - (2022-07-22)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
perceval 0.20.0-rc.1 - (2022-07-21)
**New features:**
* [phabricator] Skip a list of items by id\
Perceval will not fetch the items set with the `--blacklist-ids`
argument. In the following example, `perceval` will skip the items
with ids `123` and `456`: perceval phabricator <PHAB_URL> -t
<API_TOKEN> --blacklist-ids 123 456
Perceval 0.19.1 - (2022-06-28)
**Bug fixes:**
* [confluence] KeyError exception when arguments not passed to 'fetch_items'\
Required arguments that weren't passed to 'fetch_items' made the code
fail with a KeyError exception.
Perceval 0.19.0 - (2022-06-24)
**New features:**
* [confluence] Option `max_contents` to limit the number of requested contents\
This parameter sets the maximum number of contents requested per
query. By default, this number is 200, but this can be changed by
adding the `--max-contents` argument. This will help to bypass those
instances where the maximum limit is lower than Perceval's default
value.
**Bug fixes:**
* [bugzillarest] Failback when checking custom instances\
Some Bugzilla instances return a HTTP 400 error when checking if their
are custom instances or not. On those cases, the backend will capture
the error and consider the version of that Bugzilla instance as
custom.
* Slack identifier extraction fixed\
Extract the identifier without user information when the user was
deleted.
perceval 0.18.0 - (2022-06-01)
**New features:**
* [bugzillarest] API Key authentication\
A new authentication is available in the `bugzillarest` backend using
an API Key. This can be provided using the parameter `--api-key` on
the command line. Note that this parameter will invalidate `--backend-
user` `--backend-password`, and `--api-token` parameters. For
developers, this parameter is also available during the initialization
of the class `BugzillaRESTClient` under the name `api_key`.
* Confluence supports fetch by spaces\
This allows fetching confluence by specific spaces. By default it will
fetch the entire instance, but adding the `--spaces` argument will
fetch only these spaces.
**Feature removals:**
* Drop Python 3.6 support\
Python 3.6 reached the end of life at the end of 2021. This means it
won't receive new updates or patches to fix security issues.
Therefore, this package will only work with Python >= 3.7 from now on.
Perceval 0.15.0 - (2020-06-13)
**New features:**
* Add Rocket.Chat backend (543)\
Added support to fetch messages from a Rocket.Chat channel. The
messages are fetched in an ascending order based on time when they
were last updated. The channel information is also fetched. The tests
have been added accordingly. The usage docs have been updated.
Perceval 0.12 - (2018-10-02)
** New features and improvements: **
* So far, the JSON items written to the defined output (standard output
by default) were difficult to parse. With the option `--json-line`, each
item will be written in one line, making easier their consumption by
other processes.
* New set of backends added:
- **GoogleHits**
- **Twitter**
* Minor bugs were fixed and test coverage was improved.
** Backend improvements: **
* **gitlab**
- add merge request category
* **github**
- increase the number of items retrieved per page
- add the list of commits hashed included in pull requests
* **mediawiki**
- optimize the number of API calls
* **pipermail**
- disable SSL verification
Perceval 0.11 - (2018-05-21)
** New features improvements: **
* Problems with namespaces were fixed. This package was not really using
Python namespaces. When other packages, such as `perceval-mozilla` or
`perceval-opnfv`, were installed `__init__.py` (inside `perceval`)
were overwritten breaking the structure of the main package and making
Perceval unusable. This release defines `perceval` as a namespace. Due
to it, `fetch`, `find_backends` and other symbols are no longer accessible
from the main package.
* Mattermost backend added.
Perceval 0.10 - (2018-04-11)
** New features and improvements: **
* Support for Python 3.5, 3.6.
* New set of backends added:
- **GitLab**
- **Launchpad**
* `Cache` was removed in favor of `Archive`. This new feature stores, in
SQLite databases, each data response received from a remote source. Thus,
it is possible to retrieve original data again without accessing the remote
source.
* A new generic HTTP client (`HttpClient`) is available and shared by those
backends which require to fetch data using that protocol. This client manages
rate limits, sleep times and retries in case of error. It is fully extensible
and configurable.
* With the integration of categories, backends would be able to generate
different types of items. For instance, GitHub generates issue and
pull request items. The option `--category` allows to set which type of
items will be fetched.
* Gmane site shut down its activity in July 2016. Although there were some
actions to revamp it, it is still down. For these reasons, Gmane backend
is no longer maintained and has been removed from the core backends.
* Tests were improved, specially, adding unit tests for Gerrit backend.
* Perceval and GrimoireLab project are now part of CHAOSS community.
** Backend improvements: **
* **askbot**
- add data about accepted answers
* **gerrit**
- rename parameter URL to hostname
* **git**
- add `to-date` option to fetch data up to the given date
- run Git commands setting HOME environment variable
- clone data into a bare repository instead of a work copy
* **github**
- fetch issue comments
- fetch issue/comments reactions
- fetch multiple assignees
- fetch pull request category
- major refactoring reducing the number of requests sent by the client
* **phabricator**
- include project/user information in task transactions
** Bugs fixed: **
* The process for discovering references in Git repositories failed
with those repositories which do not have any. (260)
* When a local Git repository was analyzed by Perceval, the directory where
it was cloned was created inside the local repository. (262)
* Sleep times when rate limit is in use were wrongly calculated in some
cases, generating negative values. (355)
* Pipermail backend failed on inaccessible archive URLs. Now, it skips
those URLs generating warning messages. (358)
** Thanks to: **
* Anvesh Chaturvedi
* David Pose Fernández
* Prabhat Sharma
Perceval 0.9 - (2017-07-17)
** New features and improvements: **
* DockerHub added as new backend.
* Fetch the latest commits added in a Git repository using
the argument `latest-items`.
** Bugs fixed: **
* In Slack, comment messages were not processed raising an error
when their UUIDs were computed. These messages do not include a
'user' field on the top layer, which made the backed to fail.
This field can be found inside 'comment' key.
* Some versions of gerrit return number review as an integer.
This value must be converted to string because UUIDs can only be
generated using string values. (144)
Perceval 0.8 - (2017-05-15)
** New features and improvements: **
* Common functions used across GrimoireLab projects have been moved
to their own package. This package was named `grimoirelab-toolkit`.
From this version, Perceval depends on this package.
** Backend improvements: **
* **askbot**
- support new URLs schema for comment queries
* **bugzilla**
- set `User-Agent` header in HTTP clients
* **confluence**
- add content URL to each item
* **gerrit**
- add option to disable SSH host keys checks
* **nntp**
- raise `ParseError` exceptions when an encoding error is found
* **rss**
- set `User-Agent` header in HTTP clients
Perceval 0.7 - (2017-03-21)
** New features and improvements: **
* New set of backends added:
- Hyperkitty
- NNTP
- Slack
* `RateLimitError` exception added for handling rate limit errors.
* Code was cleaned to follow most of the PEP8 style guidelines.
** Backend improvements: **
* **git**
- retry calls on SSH commands were added to avoid temporal server errors
* **github**
- HHTP 404 errors are managed when user's organizations are fetched
- generic `RateLimitError` exception is used
** Bugs fixed: **
* In Mediawiki backend, the log messages written when a revision is not
found were set to ERROR when the real level should have been WARNING.
* The URL used to fetch jobs in Jenkins was not common to all servers.
* When UUIDs are generated with some input data, some errors may be raised
due to problems encoding invalid characters on the input. To avoid these
problems, a surrogate escape control error has been set when data is
encoded to UTF8. (123)
* Handle Meetup requests rate limit. (126)
Perceval 0.6 - (2017-02-02)
** Backend improvements: **
* **bugzillarest**
- messages in client errors were improved
* **git**
- new method `is_detached()`
* **mbox**
- ignore messages with invalid dates on `Date` header
* **phabricator**
- retry requests on HTTP 502 and 503 errors
** Bugs fixed: **
* The `mbox` class from Python's `mailbox` module fails when it tries to
decode non-ascii unix-from headers. This header is used as a separator
between messages. When this error is found, the class stops reading messages
from the mbox. Wrapping `mbox` class to override the way messages are read
was needed to catch the exception and decode the header using UTF-8.
* When a user does not exists on Phabricator, the API does not return an
error. It returns an empty list. The case where an empty list is returned
was not managed by the parser, which raised exceptions.
* In gerrit, the identifier of the change, `Change-Id` (or `id`), is not unique.
What it is unique in a gerrit sever is the number of each change and review.
This `number` is used now instead of `id` as the identifier of a review
* When Git repositories are reset to the current status on upstream, some of
them cannot deal with `origin` reference because it is ambiguous. Replacing
it by `FETCH_HEAD` works on those repositories with defined branches on
the origin.
* Git repositories in detached state do not need to be reset after `git fetch`
is called. This call is now skipped when a repository in this state
is in use. (105)
Perceval 0.5 - (2017-01-17)
** New features and improvements: **
* New set of backends added:
- Askbot
- Meetup
- RSS
* Definition of `perceval.backends` namespace and dynamic loading of backends.
These two features allow to have third party backends or packages of
backends that can be imported and used at runtime.
* Mozilla's backends were moved to their own package: `perceval-mozilla`.
* Commands were refactorized generalizing their usage into `BackendCommand`
class which can run any type of backend. This was possible thanks to the
creation of `BackendCommandArgumentParser` class, that defines, manages and
parses those arguments needed to run a command; the definition of `pre_init()`
and `post_init()` methods during the initialization of the instance; and to
the implementation of `setup_cache()` as a public function of the `cache`
module.
** Backend improvements: **
* **bugzilla**
- set maximum number of bugs requested on CSV queries
* **git**
- parse commit trailers
- new methods `is_empty()` and `count_objects()`
- set missing encodings for the command output
- cleaning up of the module
* **jenkins**
- ignore invalid job builds
* **supybot**
- parse action and bot empty lines
- parse user actions with the format `*nick msg`
- generate item ids using the body of the message
** Bugs fixed: **
* The field 'timestamp' on metadata was not generated in UTC. The call
to `datetime.now()` does not generate a timestamp in UTC. It does using
the timezone of the system. The right way is to call to `datetime.utcnow()`
method. (92)
* The docker image for Perceval purged the git package after installing
`perceval`. This made impossible to run the backend for Git because
Perceval needs of `git` command under the hood. (95)
* Git empty repositories threw errors while fetching commits. Those were
raised because on empty repositories, those which do not have any history
or are only initialized, there are some commands that cannot be run, like
a pull or log. If any of this commands is called an error is be returned.
It was fixed checking whether the repository is empty and returning
an empty list of commits for those cases. (102 and 107)
Perceval 0.4 - (2016-11-03)
** New features and improvements: **
* `category` field was added to items metadata to classify the type of
the item generated with each backend.
* The `tag` attribute added to the backends allows to mark the items
with a custom label.
* Two class methods, `has_caching` and `has_resuming`, are part now
of `Backend` class interface to notify whether a backend supports
caching and/or resuming of items.
** Backend improvements: **
* **jenkins**
- support blacklist of jobs
* **mediawiki**
- use API pages methods by default
* **phabricator**
- fetch and include projects data assigned to each task
* **redmine**
- fetch and include users data
* **remo**
- support new version of the API
* **supybot**
- parse messages written by special bots
** Bugs fixed: **
* Filepaths on merge commits were not captured on Git backend. This was
neccesary in those cases where merge commits only include data about
lines added and removed because the filepaths were not parsed and
included on the item data. (63)
* The `url` argument on the Gerrit backend was set to optional. It is
mandatory. Thus, it was set to positional on the argument parser. (60)
* Newer versions of Phabricator fixed a bug on API Conduit regarding
'constraints' parameter. The Phabricator client was modified to fix
this bug, too. (80)
* Python's `requests` library decompresses gzip-encoded responses, but
in some cases is only able to decompress some parts due to encoding
issues or to mixed contents. This problem was fixed downloading and
storing the orinal/raw data (compressed or decompressed) for furthed
processing.
* Jira backend did not return items in order, from oldest to newest. (89)
* Dates with invalid timezones were not parsed. In those cases, the
the dates will be converted usin UTC by default. (73)
Perceval 0.3 - (2016-09-19)
** New features and improvements: **
* New set of backends added:
- Phabricator
- Redmine
* Add support for creating PyPi packages
** Backend improvements: **
* **jira**
- fetch additional information about custom fields
* **mediawiki**
- add a flag which ignores the MAX_RECENT_DAYS constraint when the
backend is tested
** Bugs fixed: **
* Cache tests for Redmine backend checked the values retrieved from the
repository but not from the cache.
* Timestamps generated to fetch data from a given date included invalid
timezone information for Mediawiki API (>=1.27). It only works with Zulu
dates. (54)
* Date strings that included information after the timezone were not parsed:
`Thu, 14 Aug 2008 02:07:59 +0200 CEST`. (57)
Perceval 0.2 - (2016-07-20)
** New features and improvements: **
* New set of backends added:
- Bugzilla (REST API)
- Confluence
- Discourse
- Gmane
- Jenkins
- Kitsune (Mozilla)
- Mediawiki
- Pipermail
- ReMo (Mozilla)
- Supybot
- Telegram
* The origin of the fetched data is configurable.
* Unit tests for GitHub, Jira and Stack Exchange were added. Other tests
were added and improved. Now, the unit tests framework covers a 83% of
the source code.
** Backend improvements: **
* **gerrit**
- support server version 2.8
* **git**
- filtering by branches
- so far, the full log was read before parsing it; now, it is parsed and processed
while is being read
* **github**
- full control of GitHub API rate limit
- the program can be sent to sleep until the rate limit is reset again
* **mbox**
- fetches messages since a given date
* **pipermail**
- fetches messages from a *mod_mbox* interface (i.e: Apache)
** Bugs fixed: **
* Dates that included parentheses sections were not parsed:
`2005 15:20:32 -0100 (GMT+1)`.
* An encoding error was raised when `version.py` module was imported. (32)
* The call in chain of functions `utcnow()` and `timestamp()` from the
module `time`, produced wrong timezones on the GitHub backend.
* Action IRC messages (leading with a single `*`) were ignored. (48)
* The `backoff` field received in a Stack Exchange API response was
ignored. When this field is set, any client must wait the number of
seconds specified on it before sending any new request.
* The query used in Gerrit to retrieve the reviews was badly formed when
the blacklist filter contained two or more reviews. (50)
Perceval 0.1 - (2016-03-30)
** New features and improvements: **
* Supports Python 3.4 and newer versions.
* Fetches and caches information from several software repositories:
- Bugzilla
- Gerrit
- Git
- GitHub
- Jira
- MBox
- Stack Exchange
* Metadata fields are added to fetched items.
* Dates and times used to request data are always converted to UTC.
* Unit testing framework is available. Currently, these
unit tests cover the 62% of the source code.
** Bugs fixed: **
* Some Git commit log entries may not contain information about files.
Before this was fixed, Perceval raised an exception with a "_Unexpected
end of log stream_" message. (8)
* Empty Git commit logs raised erros when were parsed. (17)
* Character ^M) produced some parsing errors in the Git backend. (21)
Releases
sirmordred 1.1.4 - (2024-12-11)
* Update Poetry's package dependencies
sirmordred 1.1.3 - (2024-11-13)
**Bug fixes:**
* Panels import bug and Micro Mordred failure\
A bug was causing Panels to fail to import in Kibiter and preventing
micro Mordred from working. The issue was in tasks that didn’t require
a SortingHat client to run.
sirmordred 1.1.2 - (2024-10-15)
* Update Poetry's package dependencies
sirmordred 1.1.1 - (2024-09-23)
**Dependencies updateds:**
* Python minimum version updated\
Python 3.8 will reach its end of life in October 2024. Python 3.9 is
the minimum version required by the project.
sirmordred 1.1.0 - (2024-08-30)
**Performance improvements:**
* Reduced the number of connections to SortingHat\
Mordred makes a lot of connections to the SortingHat server which
could cause the uWSGI queue to fill up. When the uWSGI queue is full,
Mordred cannot connect to the SortingHat server.
sirmordred 1.0.5 - (2024-08-13)
* Update Poetry's package dependencies
sirmordred 1.0.4 - (2024-08-09)
**Bug fixes:**
* Identity refresh bug for some items\
Fix issue where some fields were not refreshed. The bug occurred
because the queries to OpenSearch filtered items in which
`author_uuid` was the individual pk, but not when the `<role>_uuid`
was the individual pk. That causes many items that contain the
`<role>_uuid` but not the `author_uuid` not to be refreshed.
sirmordred 1.0.3 - (2024-08-02)
**Bug fixes:**
* Days in collection spent time\
Include days in logs when the collection spent time is more than one
day.
sirmordred 1.0.2 - (2024-06-21)
* Update Poetry's package dependencies
sirmordred 1.0.1 - (2024-05-09)
* Update Poetry's package dependencies
sirmordred 1.0.0 - (2024-04-13)
**New features:**
* First major release\
GrimoireLab reached a stable status. This is our first major release.
sirmordred 0.10.11 - (2024-03-27)
* Update Poetry's package dependencies
sirmordred 0.10.10 - (2024-03-12)
* Update Poetry's package dependencies
sirmordred 0.10.9 - (2024-03-01)
* Update Poetry's package dependencies
sirmordred 0.10.8 - (2024-02-19)
* Update Poetry's package dependencies
sirmordred 0.10.7 - (2024-02-08)
* Update Poetry's package dependencies
sirmordred 0.10.6 - (2024-02-01)
* Update Poetry's package dependencies
sirmordred 0.10.5 - (2024-01-30)
* Update Poetry's package dependencies
sirmordred 0.10.4 - (2023-12-19)
* Update Poetry's package dependencies
sirmordred 0.10.3 - (2023-11-28)
* Update Poetry's package dependencies
sirmordred 0.10.2 - (2023-11-17)
* Update Poetry's package dependencies
sirmordred 0.10.1 - (2023-11-06)
**Bug fixes:**
* Periodic autorefresh not working\
Periodic refresh of identities was not working for backend sections
that included `:` in their name.
sirmordred 0.10.0 - (2023-10-20)
**New features:**
* Refresh identities in a shorter period\
This update significantly reduces the time required for identity
refresh operations on large projects.
**Bug fixes:**
* Missing database parameters for SortingHat\
SortingHat database parameters were not included during the
initialization of the enrich backend.
**Breaking changes:**
* Log format updated with the Mordred project name\
Update the log format of SirMordred and MicroMordred to include the
name of the project.
**Dependencies updateds:**
* Add Python 3.9 and 3.10 and drop 3.7 support\
Python 3.7 reached the end of life phase on June 27 2023 and is no
longer supported.
sirmordred 0.9.3 - (2023-08-06)
* Update Poetry's package dependencies
sirmordred 0.9.2 - (2023-07-23)
* Update Poetry's package dependencies
sirmordred 0.9.1 - (2023-07-11)
* Update Poetry's package dependencies
sirmordred 0.9.0 - (2023-06-28)
**Performance improvements:**
* Refreshing identities performance\
Decrease the time required to execute the autorefresh process by
reducing the number of queries to SortingHat.
sirmordred 0.8.0 - (2023-05-17)
**New features:**
* Optional configuration for SortingHat\
Include an optional configuration for SortingHat to define the name of
the tenant when multi-tenancy is enabled.
**Bug fixes:**
* Race condition initializing SirMordred\
There was a race condition when SirMordred initializes. The thread
that read the list of projects didn't have time to even start before
the manager finalizes it, so data backends didn't have any have any
input to work with. This bug was originally reported by ncsibra-lab49
on
[grimoirelab585](https://github.com/chaoss/grimoirelab/issues/585).
sirmordred 0.7.2 - (2023-04-28)
* Update Poetry's package dependencies
sirmordred 0.7.1 - (2023-04-27)
* Update Poetry's package dependencies
sirmordred 0.7.0 - (2023-04-21)
**New features:**
* SirMordred logs available on console\
All SirMordred logs are now available on the console output. The
`logs_dir` parameter in the `general` section is optional and it is
only needed if you also want the logs in a file.
* SortingHat option to verify SSL\
Include `verify_ssl` option to SortingHat configuration to verify the
connection with the server. By default it is `True`.
sirmordred 0.6.1 - (2023-02-03)
* Update Poetry's package dependencies
sirmordred 0.6.0 - (2023-02-01)
**Bug fixes:**
* SortingHat mandatory parameters\
Include SortingHat 0.8.x mandatory parameters. Now `sh_port`,
`sh_path`, `sh_ssl` are available to configure SortingHat.
**Breaking changes:**
* Support of SortingHat 0.8\
GrimoireLab starts using a new version of SortingHat. SortingHat is
now a web service that provides an API to manage identities and
organizations. SirMordred now connects to the new SortingHat database
using GraphQL. The previous version of SortingHat is now deprecated.
sirmordred 0.5.1 - (2022-11-07)
* Update Poetry's package dependencies
sirmordred 0.5.0 - (2022-10-31)
**New features:**
* New default GitHub pull requests alias names\
GitHub pull_request category (github:pull) has its own aliases now:
`github_pull_requests` and `github_pull_requests-raw`. This category
used to share the default aliases for GitHub: `github` and `github-
raw`.
sirmordred 0.4.0 - (2022-09-26)
**New features:**
* Configurable study names and aliases\
Study aliases and names can be set in the configuration file using the
keyword `studies_aliases`. The following snippet shows an example of
configuration. i.e. In this `aliases.json` file the name of the study
is `enrich_onion` and the name of the alias is `all_onion`
"studies_aliases": { "enrich": [ { "alias":
"all_onion", "name": "enrich_onion" }, ... ] }
**Bug fixes:**
* [git] pair-programming activation fixed\
Git pair-programming can be activated if the data source contains a
tag like `[git:pair]`
* Study selection fixed\
This fix solves the problem of executing the studies with no
associated aliases.
sirmordred 0.4.0-rc.1 - (2022-09-26)
**New features:**
* Configurable study names and aliases\
Study aliases and names can be set in the configuration file using the
keyword `studies_aliases`. The following snippet shows an example of
configuration. i.e. In this `aliases.json` file the name of the study
is `enrich_onion` and the name of the alias is `all_onion`
"studies_aliases": { "enrich": [ { "alias":
"all_onion", "name": "enrich_onion" }, ... ] }
**Bug fixes:**
* [git] pair-programming activation fixed\
Git pair-programming can be activated if the data source contains a
tag like `[git:pair]`
* Study selection fixed\
This fix solves the problem of executing the studies with no
associated aliases.
Sir Mordred 0.3.1 - (2022-06-28)
**Others:**
* GrimoireLab dependencies update\
Dependencies have been updated to use the latest versions of
GrimoireLab packages:
- grimoirelab-toolkit >= 0.3
- graal >= 0.3
- cereslib >= 0.3
- kidash >= 0.5
- grimoirelab-penales >= 0.1
- grimoire-elk >= 0.102
Perceval dependencies are removed because they are installed
with `grimoire-elk`.
Sir Mordred 0.3.0 - (2022-06-07)
**New features:**
* Dependencies file for development\
Include a new requirements.txt file to install the development version
from the repositories.
* Add extract spaces\
Extract spaces from the URL. By default it will extract labels but
adding `tag_type="spaces"` it will extract spaces.
**Bug fixes:**
* Studies aliases relocated\
Relocate studies aliases so they are not created automatically by
Mordred.
* Add pair-programming in '_get_enrich_backend' method\
Add the `pair-programming` to the `_get_enrich_backend` method if it
exists in the setup.cfg file.
**Feature removals:**
* Drop Python 3.6 support\
Python 3.6 reached the end of life at the end of 2021. This means it
won't receive new updates or patches to fix security issues.
Therefore, this package will only work with Python >= 3.7 from now on.
* FINOS Meeting backend removed\
FINOS Meeting backend is not supported anymore. The Perceval backend
was archived in Bitergia repository so it shouldn't be part of the
core anymore because it's software not maintained.
**Others:**
* Add studies to the alias file\
Add onion, areas of code, and demographics studies to the file
aliases.json. This is needed to keep this file updated with the
aliases we use in the dashboards by adding them to the appropiate data
sources. The included aliases: - all_onion - git_areas_of_code -
demographics
Releases
perceval-opnfv 1.0.10 - (2024-12-11)
* Update Poetry's package dependencies
perceval-opnfv 1.0.9 - (2024-11-13)
* Update Poetry's package dependencies
perceval-opnfv 1.0.8 - (2024-10-15)
* Update Poetry's package dependencies
perceval-opnfv 1.0.7 - (2024-09-23)
**Dependencies updateds:**
* Python minimum version updated\
Python 3.8 will reach its end of life in October 2024. Python 3.9 is
the minimum version required by the project.
perceval-opnfv 1.0.6 - (2024-08-30)
* Update Poetry's package dependencies
perceval-opnfv 1.0.5 - (2024-08-13)
* Update Poetry's package dependencies
perceval-opnfv 1.0.4 - (2024-08-09)
* Update Poetry's package dependencies
perceval-opnfv 1.0.3 - (2024-08-02)
* Update Poetry's package dependencies
perceval-opnfv 1.0.2 - (2024-06-21)
* Update Poetry's package dependencies
perceval-opnfv 1.0.1 - (2024-05-09)
* Update Poetry's package dependencies
perceval-opnfv 1.0.0 - (2024-04-13)
**New features:**
* First major release\
GrimoireLab reached a stable status. This is our first major release.
perceval-opnfv 0.2.25 - (2024-03-27)
* Update Poetry's package dependencies
perceval-opnfv 0.2.24 - (2024-03-12)
* Update Poetry's package dependencies
perceval-opnfv 0.2.23 - (2024-03-01)
* Update Poetry's package dependencies
perceval-opnfv 0.2.22 - (2024-02-08)
* Update Poetry's package dependencies
perceval-opnfv 0.2.21 - (2024-01-30)
* Update Poetry's package dependencies
perceval-opnfv 0.2.20 - (2023-12-19)
* Update Poetry's package dependencies
perceval-opnfv 0.2.19 - (2023-11-28)
* Update Poetry's package dependencies
perceval-opnfv 0.2.18 - (2023-11-14)
* Update Poetry's package dependencies
perceval-opnfv 0.2.17 - (2023-11-03)
* Update Poetry's package dependencies
perceval-opnfv 0.2.16 - (2023-10-20)
**Dependencies updateds:**
* Add Python 3.10 and 3.11 and drop 3.7 support\
Python 3.7 reached the end of life phase on June 27 2023 and is no
longer supported.
perceval-opnfv 0.2.15 - (2023-08-06)
* Update Poetry's package dependencies
perceval-opnfv 0.2.14 - (2023-07-23)
* Update Poetry's package dependencies
perceval-opnfv 0.2.13 - (2023-07-11)
* Update Poetry's package dependencies
perceval-opnfv 0.2.12 - (2023-06-28)
* Update Poetry's package dependencies
perceval-opnfv 0.2.11 - (2023-05-17)
* Update Poetry's package dependencies
perceval-opnfv 0.2.10 - (2023-04-27)
* Update Poetry's package dependencies
perceval-opnfv 0.2.9 - (2023-04-26)
* Update Poetry's package dependencies
perceval-opnfv 0.2.8 - (2023-04-21)
* Update Poetry's package dependencies
perceval-opnfv 0.2.7 - (2023-02-03)
* Update Poetry's package dependencies
perceval-opnfv 0.2.6 - (2023-02-01)
* Update Poetry's package dependencies
perceval-opnfv 0.2.5 - (2022-11-07)
* Update Poetry's package dependencies
perceval-opnfv 0.2.4 - (2022-10-31)
* Update Poetry's package dependencies
perceval-opnfv 0.2.1 - (2022-06-24)
**Others:**
* Update GrimoireLab dependencies\
Dependencies have been updated to use the latest versions
of GrimoireLab packages:
- grimoirelab-toolkit >= 0.3
- perceval >= 0.19
perceval-opnfv 0.2.0 - (2022-06-02)
**New features:**
* Dependencies file for development\
Include a new requirements.txt file to install the development version
from the repositories.
Releases
grimoire-elk 1.3.2 - (2024-12-11)
* Update Poetry's package dependencies
grimoire-elk 1.3.1 - (2024-11-13)
* Update Poetry's package dependencies
grimoire-elk 1.3.0 - (2024-10-15)
**New features:**
* Keywords data on enriched items\
Keywords field is included now on the enriched items of bugzilla and
bugzillarest indices.
* New reponse times on bugzilla items\
The Bugzilla enriched items include two new fields to track response
times on these type of repositories. The field
`time_to_first_attention` is the the time expressed in days between
the ticket creation and the first comment from a contributor different
from the author creating the bug. The field `last_comment_date` is the
date of the last comment posted in the bug.
**Bug fixes:**
* All references processed for the reference analysis study\
The `enrich_reference_analysis` study analyzes the cross-references
between "issues" and "pull request". When we use an aggregations
query, it returns only the first 10 items (ElasticSearch/OpenSearch by
default). By using 'composite aggregations', we can paginate the
result and thus, obtain all the references.
grimoire-elk 1.2.0 - (2024-09-23)
**New features:**
* Status in SUMO enriched index\
Add the missing status in the SUMO backend. The status could be
`solved`, `archived`, `locked` or `open`.
**Bug fixes:**
* Avoid UNDEFINED values in author fields\
Avoid `-- UNDEFINED --` values for all SortingHat fields when Mordred
loses connection to the SortingHat server during the enrichment or
autorefresh execution. It will keep the values of the `name`, `email`,
`id` and `uuid` fields.
* Handle OpenSearch page info on pagination\
In OpenSearch and ElasticSearch < 7.x the page info on pagination is
different. This will handle both of them.
**Dependencies updateds:**
* Python minimum version updated\
Python 3.8 will reach its end of life in October 2024. Python 3.9 is
the minimum version required by the project.
grimoire-elk 1.1.5 - (2024-08-30)
**Bug fixes:**
* Git branches study improved\
Previously, while the branches study was running, the `branches` field
remained empty or partially filled until the study was completed,
leading to incorrect data being displayed on the dashboard. With this
change, the branches field is updated only after the study has
finished.
grimoire-elk 1.1.4 - (2024-08-13)
**Bug fixes:**
* Roles attribute not available on some enrichers\
The 'roles' attribute wasn't available for all the enrichers. The
attribute has been added to the main class 'Enrich' so subclasses will
have it available even when they don't use it.
grimoire-elk 1.1.3 - (2024-08-09)
**Bug fixes:**
* GitHub URL ending with slash failed (1159)\
GrimoireLab failed to run with GitHub URLs ending in a slash
* Identity refresh bug for some items (1161)\
Update the filter construction to correctly match UUIDs in OpenSearch,
addressing a bug in Mordred that prevented role identities from being
refreshed.
grimoire-elk 1.1.2 - (2024-08-02)
**Bug fixes:**
* Git branches study fixed\
Git branches study was assigning branches to wrong commits.
grimoire-elk 1.1.1 - (2024-06-21)
**Bug fixes:**
* Kitsune mapping updated\
Kitsune raw mapping was incorrectly defined with `dynamic: false` in
the data attribute.
grimoire-elk 1.1.0 - (2024-05-09)
**New features:**
* Kitsune fields updated\
Include new fields in Kitsune backend enriched index. Include
`product` and `url` fields in answers, and
`is_kitsune_question_solved`, `time_to_first_reply` and `url` in
questions.
grimoire-elk 1.0.0 - (2024-04-13)
**New features:**
* First major release\
GrimoireLab reached a stable status. This is our first major release.
**Bug fixes:**
* MBOX origin URL anonymized\
Remove credentials from MBOX origin URL and tag fields.
grimoire-elk 0.111.1 - (2024-03-27)
**Bug fixes:**
* Kitsune inmerse terms error\
Update raw mapping of Kitsune indexes to handle immense terms found in
`data.answers_data.content`, and `data.content`.
grimoire-elk 0.111.0 - (2024-03-12)
**New features:**
* Kitsune demography study\
Include demography study in Kitsune (SUMO). And update the index to
include standard fields such as a unique identifier (`id`) and some
missing fields like `origin` or `uuid`.
grimoire-elk 0.110.0 - (2024-03-01)
**New features:**
* Kitsune raw mapping updated\
Update the mapping for Kitsune backend to fix an error inserting
Perceval items in the raw index when the field 'data.metadata.value'
is too big.
grimoire-elk 0.109.8 - (2024-02-19)
* Update Poetry's package dependencies
grimoire-elk 0.109.7 - (2024-02-08)
**Bug fixes:**
* Bug on some backends enrichment\
Mediawiki, Meetup are Jira were failing to retrieve the enrollment for
an identity.
grimoire-elk 0.109.6 - (2024-02-01)
* Update Poetry's package dependencies
grimoire-elk 0.109.5 - (2024-01-30)
* Update Poetry's package dependencies
grimoire-elk 0.109.4 - (2023-12-19)
* Update Poetry's package dependencies
grimoire-elk 0.109.3 - (2023-11-28)
* Update Poetry's package dependencies
grimoire-elk 0.109.2 - (2023-11-17)
* Update Poetry's package dependencies
grimoire-elk 0.109.1 - (2023-11-03)
* Update Poetry's package dependencies
grimoire-elk 0.109.0 - (2023-10-20)
**New features:**
* Anonymize repository URLs in graal backend\
The raw data generated by the graal backend had non-anonymized URLs on
the meta fields 'origin' and 'tag'. This was necessary to generate
enriched data for non-public repositories.
* Bug description in Bugzilla\
Include the description of the bugs in Bugzilla. The new fields are
'description' and 'description_analyzed'. The second allows to query
any of the words of the description.
* SortingHat backend name per data source\
Allow to define the name of the backend for the identities stored in
SortingHat. From now on github2 is renamed as github in SortingHat
identities.
**Bug fixes:**
* Individual `id` incorrectly assigned\
Correct a bug that results in enriched items using the individual ID
as the identity ID, and vice versa.
* Error in OpenSearch response stops demography study\
Fix an issue that stops demography study when OpenSearch occasionally
provides a response without some expected fields.
* Undefined identities in GitHub comments\
Fix a bug that causes certain identities from commentaries to not be
imported into SortingHat, resulting in them appearing as UNDEFINED in
OpenSearch.
**Dependencies updateds:**
* Add Python 3.9 and 3.10 and drop 3.7 support\
Python 3.7 reached the end of life phase on June 27 2023 and is no
longer supported.
* Bump urllib3 version\
Bump urllib3 from ^1.26 to ^2.0.7
* Remove Python pymsql dependency\
Remove the dependency with PyMySQL because it is not used anymore.
grimoire-elk 0.108.1 - (2023-08-06)
* Update Poetry's package dependencies
grimoire-elk 0.108.0 - (2023-07-23)
**Bug fixes:**
* Enrich items with multiple enrollments\
The `date` field must be a string since improved performance by
decreasing calls to the `str_to_datetime` method.
**Performance improvements:**
* Load identities performance\
Improve the load identities process performance by reducing the number
of queries to SortingHat.
grimoire-elk 0.107.0 - (2023-07-11)
**Performance improvements:**
* Enrichment processing time reduced by 50%\
The general performance was improved reducing the number of calls to
the identities manager (i.g. `SortingHat`). There were some deprecated
calls that weren't needed any longer and also, we increased the cache
of individuals in ELKs. We were also able to reduce the processing
time of the Git backend by converting commit dates only once.
grimoire-elk 0.106.0 - (2023-06-28)
**Performance improvements:**
* Refreshing identities performance\
Decrease the time required to execute the autorefresh process by
reducing the number of queries to SortingHat.
grimoire-elk 0.105.0 - (2023-05-17)
**Breaking changes:**
* SortingHat configuration\
When SortingHat's multi-tenancy is enabled, the tenant is included in
all clients during execution.
grimoire-elk 0.104.6 - (2023-04-28)
* Update Poetry's package dependencies
grimoire-elk 0.104.5 - (2023-04-27)
* Update Poetry's package dependencies
grimoire-elk 0.104.4 - (2023-04-26)
* Update Poetry's package dependencies
grimoire-elk 0.104.3 - (2023-04-21)
**Bug fixes:**
* Fix typo in SortingHat fields\
Some fields were not updated from the old version of SortingHat. Now
the API returns 'isBot' and 'genderAcc' instead of 'is_bot' and
'gender_acc'.
**Breaking changes:**
* Define date format for Git documents\
Git by default uses a custom date format that is not recognized by
Elastic/OpenSearch. Specify the format for the AuthorDate and
CommitDate fields in Git documents to support the formats usually
returned by `git log`. This allows us to do proper filtering on these
fields such as "all documents with CommitDate before 2020".
* Update support for ES/OS versions > 7.X/1.X\
Currently the support for ES/OS versions > 7.X/1.X is broken due to a
faulty version check. Add a static method to the Elasticsearch object
that correctly checks whether the version is 'legacy' (defined as ES
6.X or OS 0.X) and adds this check to various places in the codebase.
Additionally, fixes an instance where `interval` is used instead of
`fixed_interval`. `interval` was deprecated in ES7.
grimoire-elk 0.104.2 - (2023-02-03)
* Update Poetry's package dependencies
grimoire-elk 0.104.1 - (2023-02-01)
* Update Poetry's package dependencies
grimoire-elk 0.104.0 - (2023-02-01)
**Bug fixes:**
* SortingHatClientError exception fixed\
Exception raised with SortingHatClientError has been fixed in
add_identity method.
* SortingHat mandatory parameters\
Include SortingHat 0.8.x mandatory parameters. Now `sh_port`,
`sh_path`, `sh_ssl` are required.
**Breaking changes:**
* Support of SortingHat 0.8\
GrimoireLab starts using a new version of SortingHat. SortingHat is
now a web service that provides an API to manage identities and
organizations. GrimoireELK now connects to the new SortingHat database
using GraphQL. The previous version of SortingHat is now deprecated.
grimoire-elk 0.103.3 - (2022-11-07)
**Bug fixes:**
* Demographic study on Weblate fixed\
Weblate demographic study is now updated to allow passing the study
alias name by parameter.
grimoire-elk 0.103.2 - (2022-10-31)
**New features:**
* Development and reviewing contributors fields for Git items\
New information about the participants during the development and the
reviewing processes are available in `Git` items. The Git backend
extracts these data from `Approved-by`, `Co-authored-by`, and
`Merged-by` trailers that some commit messages might include. For each
one of these trailers, new fields will be available with the suffixes:
`_bots`, `_domains`, `_names`, `_org_names`, and `_uuids`. Also for
each one of these suffixes, there will be extra fields with the prefix
`non_authored_`. This is an example of the fields available for the
the trailer `Merged-by`: - merged_by_multi_bots -
merged_by_multi_domains - merged_by_multi_names -
merged_by_multi_org_names - merged_by_multi_uuids -
non_authored_merged_by_multi_bots -
non_authored_merged_by_multi_domains -
non_authored_merged_by_multi_names -
non_authored_merged_by_multi_org_names
* Option 'to-date' on the backend sections\
The option `to-date` on the `setup.cfg` allows fetching data until a
specific date. By default, this value is `2100-01-01`.
**Bug fixes:**
* Onion study on Github pull requests fixed\
Github pull requests onion study is now updated to allow passing the
study alias name by parameter.
* Onion study on Gitlab fixed\
Gitlab onion study is now updated to allow passing the study alias
name by parameter.
**Dependencies updateds:**
* New Pandas version\
Update Pandas to the newest version compatible with Python 3.7+.
grimoire-elk 0.103.0 - (2022-09-26)
**New features:**
* git-branches study skip uncloned repo\
The git-branches study skips the repository when it is not cloned and
continues with the next one.
* Configurable study aliases\
Before this fix, the name of the study aliases was hard coded. From
now on, these values can be configured by passing them as parameters.
* Entry points to support enriching plugin mechanism\
This plugin mechanism allows ELK to use third-party enriching
backends. Developers only need to include the next snippet in their
package configuration and ELK will import them as any other core
backend. entry_points={"grimoire_elk": "mybackend =
package.subpackage:get_connectors"}, **NOTE**: In the previous
example, `get_connectors` is a function that returns the backend
classes needed to enrich a datasource. Take the next snipped as an
example: def get_connectors(): return {"backend": [MyBackend,
MyBackendOcean, MyBackendEnrich, MyBackendCommand]}
**Bug fixes:**
* [raw/slack] Handle fields in "data.file"\
Avoid the 'Failed to insert data to ES' error when a document contains
at least one immense term in 'data.file' (whose UTF8 encoding is
longer than the max length 32766).
grimoire-elk 0.103.0-rc.10 - (2022-09-26)
**New features:**
* git-branches study skip uncloned repo\
The git-branches study skips the repository when it is not cloned and
continues with the next one.
* Configurable study aliases\
Before this fix, the name of the study aliases was hard coded. From
now on, these values can be configured by passing them as parameters.
* Entry points to support enriching plugin mechanism\
This plugin mechanism allows ELK to use third-party enriching
backends. Developers only need to include the next snippet in their
package configuration and ELK will import them as any other core
backend. entry_points={"grimoire_elk": "mybackend =
package.subpackage:get_connectors"}, **NOTE**: In the previous
example, `get_connectors` is a function that returns the backend
classes needed to enrich a datasource. Take the next snipped as an
example: def get_connectors(): return {"backend": [MyBackend,
MyBackendOcean, MyBackendEnrich, MyBackendCommand]}
**Bug fixes:**
* [raw/slack] Handle fields in "data.file"\
Avoid the 'Failed to insert data to ES' error when a document contains
at least one immense term in 'data.file' (whose UTF8 encoding is
longer than the max length 32766).
grimoire-elk 0.103.0-rc.9 - (2022-09-26)
**New features:**
* Configurable study aliases\
Before this fix, the name of the study aliases was hard coded. From
now on, these values can be configured by passing them as parameters.
* git-branches study skip uncloned repo\
The git-branches study skips the repository when it is not cloned and
continues with the next one.
* Entry points to support enriching plugin mechanism\
This plugin mechanism allows ELK to use third-party enriching
backends. Developers only need to include the next snippet in their
package configuration and ELK will import them as any other core
backend. entry_points={"grimoire_elk": "mybackend =
package.subpackage:get_connectors"}, **NOTE**: In the previous
example, `get_connectors` is a function that returns the backend
classes needed to enrich a datasource. Take the next snipped as an
example: def get_connectors(): return {"backend": [MyBackend,
MyBackendOcean, MyBackendEnrich, MyBackendCommand]}
**Bug fixes:**
* [raw/slack] Handle fields in "data.file"\
Avoid the 'Failed to insert data to ES' error when a document contains
at least one immense term in 'data.file' (whose UTF8 encoding is
longer than the max length 32766).
grimoire-elk 0.103.0-rc.8 - (2022-09-26)
**New features:**
* git-branches study skip uncloned repo\
The git-branches study skips the repository when it is not cloned and
continues with the next one.
* Configurable study aliases\
Before this fix, the name of the study aliases was hard coded. From
now on, these values can be configured by passing them as parameters.
* Entry points to support enriching plugin mechanism\
This plugin mechanism allows ELK to use third-party enriching
backends. Developers only need to include the next snippet in their
package configuration and ELK will import them as any other core
backend. entry_points={"grimoire_elk": "mybackend =
package.subpackage:get_connectors"}, **NOTE**: In the previous
example, `get_connectors` is a function that returns the backend
classes needed to enrich a datasource. Take the next snipped as an
example: def get_connectors(): return {"backend": [MyBackend,
MyBackendOcean, MyBackendEnrich, MyBackendCommand]}
**Bug fixes:**
* [raw/slack] Handle fields in "data.file"\
Avoid the 'Failed to insert data to ES' error when a document contains
at least one immense term in 'data.file' (whose UTF8 encoding is
longer than the max length 32766).
grimoire-elk 0.103.0-rc.7 - (2022-09-23)
**New features:**
* Configurable study aliases\
Before this fix, the name of the study aliases was hard coded. From
now on, these values can be configured by passing them as parameters.
* git-branches study skip uncloned repo\
The git-branches study skips the repository when it is not cloned and
continues with the next one.
* Entry points to support enriching plugin mechanism\
This plugin mechanism allows ELK to use third-party enriching
backends. Developers only need to include the next snippet in their
package configuration and ELK will import them as any other core
backend. entry_points={"grimoire_elk": "mybackend =
package.subpackage:get_connectors"}, **NOTE**: In the previous
example, `get_connectors` is a function that returns the backend
classes needed to enrich a datasource. Take the next snipped as an
example: def get_connectors(): return {"backend": [MyBackend,
MyBackendOcean, MyBackendEnrich, MyBackendCommand]}
**Bug fixes:**
* [raw/slack] Handle fields in "data.file"\
Avoid the 'Failed to insert data to ES' error when a document contains
at least one immense term in 'data.file' (whose UTF8 encoding is
longer than the max length 32766).
grimoire-elk 0.103.0-rc.6 - (2022-09-23)
**New features:**
* Configurable study aliases\
Before this fix, the name of the study aliases was hard coded. From
now on, these values can be configured by passing them as parameters.
* git-branches study skip uncloned repo\
The git-branches study skips the repository when it is not cloned and
continues with the next one.
* Entry points to support enriching plugin mechanism\
This plugin mechanism allows ELK to use third-party enriching
backends. Developers only need to include the next snippet in their
package configuration and ELK will import them as any other core
backend. entry_points={"grimoire_elk": "mybackend =
package.subpackage:get_connectors"}, **NOTE**: In the previous
example, `get_connectors` is a function that returns the backend
classes needed to enrich a datasource. Take the next snipped as an
example: def get_connectors(): return {"backend": [MyBackend,
MyBackendOcean, MyBackendEnrich, MyBackendCommand]}
**Bug fixes:**
* [raw/slack] Handle fields in "data.file"\
Avoid the 'Failed to insert data to ES' error when a document contains
at least one immense term in 'data.file' (whose UTF8 encoding is
longer than the max length 32766).
grimoire-elk 0.103.0-rc.5 - (2022-09-23)
**New features:**
* Configurable study aliases\
Before this fix, the name of the study aliases was hard coded. From
now on, these values can be configured by passing them as parameters.
* git-branches study skip uncloned repo\
The git-branches study skips the repository when it is not cloned and
continues with the next one.
* Entry points to support enriching plugin mechanism\
This plugin mechanism allows ELK to use third-party enriching
backends. Developers only need to include the next snippet in their
package configuration and ELK will import them as any other core
backend. entry_points={"grimoire_elk": "mybackend =
package.subpackage:get_connectors"}, **NOTE**: In the previous
example, `get_connectors` is a function that returns the backend
classes needed to enrich a datasource. Take the next snipped as an
example: def get_connectors(): return {"backend": [MyBackend,
MyBackendOcean, MyBackendEnrich, MyBackendCommand]}
**Bug fixes:**
* [raw/slack] Handle fields in "data.file"\
Avoid the 'Failed to insert data to ES' error when a document contains
at least one immense term in 'data.file' (whose UTF8 encoding is
longer than the max length 32766).
grimoire-elk 0.103.0-rc.4 - (2022-09-07)
**New features:**
* Configurable study aliases\
Before this fix, the name of the study aliases was hard coded. From
now on, these values can be configured by passing them as parameters.
* git-branches study skip uncloned repo\
The git-branches study skips the repository when it is not cloned and
continues with the next one.
* Entry points to support enriching plugin mechanism\
This plugin mechanism allows ELK to use third-party enriching
backends. Developers only need to include the next snippet in their
package configuration and ELK will import them as any other core
backend. entry_points={"grimoire_elk": "mybackend =
package.subpackage:get_connectors"}, **NOTE**: In the previous
example, `get_connectors` is a function that returns the backend
classes needed to enrich a datasource. Take the next snipped as an
example: def get_connectors(): return {"backend": [MyBackend,
MyBackendOcean, MyBackendEnrich, MyBackendCommand]}
**Bug fixes:**
* [raw/slack] Handle fields in "data.file"\
Avoid the 'Failed to insert data to ES' error when a document contains
at least one immense term in 'data.file' (whose UTF8 encoding is
longer than the max length 32766).
grimoire-elk 0.103.0-rc.3 - (2022-08-23)
**New features:**
* Configurable study aliases\
Before this fix, the name of the study aliases was hard coded. From
now on, these values can be configured by passing them as parameters.
* Entry points to support enriching plugin mechanism\
This plugin mechanism allows ELK to use third-party enriching
backends. Developers only need to include the next snippet in their
package configuration and ELK will import them as any other core
backend. entry_points={"grimoire_elk": "mybackend =
package.subpackage:get_connectors"}, **NOTE**: In the previous
example, `get_connectors` is a function that returns the backend
classes needed to enrich a datasource. Take the next snipped as an
example: def get_connectors(): return {"backend": [MyBackend,
MyBackendOcean, MyBackendEnrich, MyBackendCommand]}
**Bug fixes:**
* [raw/slack] Handle fields in "data.file"\
Avoid the 'Failed to insert data to ES' error when a document contains
at least one immense term in 'data.file' (whose UTF8 encoding is
longer than the max length 32766).
grimoire-elk 0.103.0-rc.2 - (2022-07-22)
**New features:**
* Entry points to support enriching plugin mechanism\
This plugin mechanism allows ELK to use third-party enriching
backends. Developers only need to include the next snippet in their
package configuration and ELK will import them as any other core
backend. entry_points={"grimoire_elk": "mybackend =
package.subpackage:get_connectors"}, **NOTE**: In the previous
example, `get_connectors` is a function that returns the backend
classes needed to enrich a datasource. Take the next snipped as an
example: def get_connectors(): return {"backend": [MyBackend,
MyBackendOcean, MyBackendEnrich, MyBackendCommand]}
* Configurable study aliases\
Before this fix, the name of the study aliases was hard coded. From
now on, these values can be configured by passing them as parameters.
**Bug fixes:**
* [raw/slack] Handle fields in "data.file"\
Avoid the 'Failed to insert data to ES' error when a document contains
at least one immense term in 'data.file' (whose UTF8 encoding is
longer than the max length 32766).
grimoire-elk 0.103.0-rc.1 - (2022-07-21)
**New features:**
* Entry points to support enriching plugin mechanism\
This plugin mechanism allows ELK to use third-party enriching
backends. Developers only need to include the next snippet in their
package configuration and ELK will import them as any other core
backend. entry_points={"grimoire_elk": "mybackend =
package.subpackage:get_connectors"}, **NOTE**: In the previous
example, `get_connectors` is a function that returns the backend
classes needed to enrich a datasource. Take the next snipped as an
example: def get_connectors(): return {"backend": [MyBackend,
MyBackendOcean, MyBackendEnrich, MyBackendCommand]}
* Configurable study aliases\
Before this fix, the name of the study aliases was hard coded. From
now on, these values can be configured by passing them as parameters.
**Bug fixes:**
* [raw/slack] Handle fields in "data.file"\
Avoid the 'Failed to insert data to ES' error when a document contains
at least one immense term in 'data.file' (whose UTF8 encoding is
longer than the max length 32766).
GrimoireELK 0.102.0 - (2022-06-24)
**Bug fixes:**
* Poetry build version updated\
The current version of the `poetry` builder was old and caused some
problems when installing packages in developer mode. With this new
version, it's possible to install ELK in developer or editable mode
without using `setuptools` files (i.e. `setup.cfg` and `setup.py`).
**Others:**
* GrimoireLab dependencies update\
Dependencies have been updated to use the latest versions of
GrimoireLab packages:
- grimoirelab-toolkit >= 0.3
- perceval >= 0.19
- perceval-mozilla >= 0.3
- perceval-opnfv >=0.2
- perceval-puppet >= 0.2
- perceval-weblate >= 0.2
- graal >= 0.3
- cereslib >= 0.3
GrimoireELK 0.101.1 - (2022-06-03)
**Bug fixes:**
* Pair programming regular expression improvement\
The regular expression to detect pair programming authors in git
datasource has been improved adding more connectors. The following
list shows the current connectors: - `[aA][nN][dD]` - `&` - `+`
* Fix fetching authors with min and max dates\
The old query only returns 10000 items due to ElasticSearch and
OpenSearch now use `composite` aggregation to paginate all buckets.
GrimoireELK 0.101.0 - (2022-06-03)
**New features:**
* Anonymize parameters\
A new API function `anonymize_params(params)` allows to anonymize a
list of parameters that developers might consider secrets. It will
only take effect for those parameters which their names are defined on
`SECRET_PARAMETERS` constant.
* Dependencies file for development\
Include a new requirements.txt file to install the development version
from the repositories.
* Support to add identities to SortingHat from Enrich class\
New methods `add_identities` and `add_identity` are available on
`Enrich` class to add new identities to a `SortingHat` database.
Libraries using `elk` won't need to use `sortinghat` library to store
new identities while enriching items.
* Collect and enrich confluence by spaces\
Users can collect and enrich Confluence data from specific spaces
only, adding the option `--spaces=[SPACE1, SPACE2, ...]` to their
`projects.json`. The following example shows how to collect and to
enrich data from only the `DEV` and `SUPP` spaces only. {
"project": { "confluence": [ "http://example.com
--spaces=[DEV, SUPP]" ] } }
* Add demographic study to mattermost\
Add the demographic study to the mattermost data source.
**Bug fixes:**
* Skip sources with wrong arguments\
Sources won't interrupt collection or enrichment processes when any of
their arguments are invalid. Instead, they will be skipped.
* Git fix 'is_git_commit_multi_author'\
This fixes the `is_git_commit_multi_author` field when the commit has
more than one author and `pair-programming` is enabled. This is
important for calculating `pair_programming_*` values.
* SortingHat as a required package\
SortingHat is required for running grimoireelk but it was set as an
extra dependency. From now on sortinghat is always installed with
grimoireelk.
* Confluence credentials not stored in raw indexes\
Credentials for Confluence datasource were stored in raw indices as
part of the URL. For now on, credentials that are part of URLs are
removed before any data is stored.
**Feature removals:**
* Drop Python 3.6 support\
Python 3.6 reached the end of life at the end of 2021. This means it
won't receive new updates or patches to fix security issues.
Therefore, this package will only work with Python >= 3.7 from now on.
* FINOS Meeting backend removed\
FINOS Meeting backend is not supported anymore. The Perceval backend
was archived in Bitergia repository so it shouldn't be part of the
core anymore because it's software not maintained.
GrimoireELK 0.75.0 - (2020-06-20)
**New features:**
* Add support for RocketChat (876)\
Added support for collecting and enriching the messages of RocketChat
instances. The schema of the enriched data has been included. Tests
have been added too. This work is a community contribution, which has
been possible thanks to Obaro Ikoh and Animesh Kumar.
GrimoireELK 0.72.0 - (2020-04-24)
**New features:**
* code complexity per language (782)\
Add language differentiation to code complexity analysis (Cocom)
* Add support for Gitter (820)\
Added support for creating raw and enriched indexes of message from
Gitter. The visualizations and tests for extracted data have also been
added.
* Add support for Pagure (821)\
Added support for creating raw and enriched indexes of Pagure. Issued
from a Pagure repository are enriched. Additionally, the comments of
these issues (if present) are also enriched. The schema for this
backend has also been added. Tests have been included accordingly.
* Store and process GitHub events\
ELK is enhanced to process and store events collected from the GitHub
API. The events currently supported are - LabeledEvent -
UnlabeledEvent - AddedToProjectEvent - MovedColumnsInProjectEvent -
RemovedFromProjectEvent - CrossReferencedEvent - ClosedEvent
**Bug fixes:**
* Fix missing geolocation in Meetup event (826)\
This fix allows to process Meetup events for which geolocation
information isn't provided. For these events, the corresponding
attribute in the enriched index (`venue_geolocation`) is set to None.
Releases
perceval-mozilla 1.0.10 - (2024-12-11)
* Update Poetry's package dependencies
perceval-mozilla 1.0.9 - (2024-11-13)
**Bug fixes:**
* Kitsune resuming feature fixed (79)\
Kitsune backend resuming feature was broken. It didn't retrieve the
updated questions right using the parameter 'offset'. Changing the
backend to use 'from-date' parameter instead has fixed the problem.
perceval-mozilla 1.0.8 - (2024-10-15)
* Update Poetry's package dependencies
perceval-mozilla 1.0.7 - (2024-09-23)
**Dependencies updateds:**
* Python minimum version updated\
Python 3.8 will reach its end of life in October 2024. Python 3.9 is
the minimum version required by the project.
perceval-mozilla 1.0.6 - (2024-08-30)
* Update Poetry's package dependencies
perceval-mozilla 1.0.5 - (2024-08-13)
* Update Poetry's package dependencies
perceval-mozilla 1.0.4 - (2024-08-09)
* Update Poetry's package dependencies
perceval-mozilla 1.0.3 - (2024-08-02)
* Update Poetry's package dependencies
perceval-mozilla 1.0.2 - (2024-06-21)
* Update Poetry's package dependencies
perceval-mozilla 1.0.1 - (2024-05-09)
* Update Poetry's package dependencies
perceval-mozilla 1.0.0 - (2024-04-13)
**New features:**
* First major release\
GrimoireLab reached a stable status. This is our first major release.
perceval-mozilla 0.4.3 - (2024-03-27)
* Update Poetry's package dependencies
perceval-mozilla 0.4.2 - (2024-03-12)
* Update Poetry's package dependencies
perceval-mozilla 0.4.1 - (2024-03-01)
* Update Poetry's package dependencies
perceval-mozilla 0.4.0 - (2024-02-19)
**New features:**
* Retry Kitsune when rate limited\
Kitsune now includes the `--sleep-for-rate` option to manage `429 Too
Many Requests` errors. You can configure retries and sleep duration
using the `--max-retries` and `--sleep-time` options respectively.
perceval-mozilla 0.3.22 - (2024-02-08)
* Update Poetry's package dependencies
perceval-mozilla 0.3.21 - (2024-01-30)
* Update Poetry's package dependencies
perceval-mozilla 0.3.20 - (2023-12-19)
* Update Poetry's package dependencies
perceval-mozilla 0.3.19 - (2023-11-28)
* Update Poetry's package dependencies
perceval-mozilla 0.3.18 - (2023-11-15)
* Update Poetry's package dependencies
perceval-mozilla 0.3.17 - (2023-11-03)
* Update Poetry's package dependencies
perceval-mozilla 0.3.16 - (2023-10-20)
**Dependencies updateds:**
* Add Python 3.10 and 3.11 and drop 3.7 support\
Python 3.7 reached the end of life phase on June 27 2023 and is no
longer supported.
perceval-mozilla 0.3.15 - (2023-08-06)
* Update Poetry's package dependencies
perceval-mozilla 0.3.14 - (2023-07-23)
* Update Poetry's package dependencies
perceval-mozilla 0.3.13 - (2023-07-11)
* Update Poetry's package dependencies
perceval-mozilla 0.3.12 - (2023-06-28)
* Update Poetry's package dependencies
perceval-mozilla 0.3.11 - (2023-05-17)
* Update Poetry's package dependencies
perceval-mozilla 0.3.10 - (2023-04-27)
* Update Poetry's package dependencies
perceval-mozilla 0.3.9 - (2023-04-26)
* Update Poetry's package dependencies
perceval-mozilla 0.3.8 - (2023-04-21)
* Update Poetry's package dependencies
perceval-mozilla 0.3.7 - (2023-02-03)
* Update Poetry's package dependencies
perceval-mozilla 0.3.6 - (2023-02-01)
* Update Poetry's package dependencies
perceval-mozilla 0.3.5 - (2022-11-07)
* Update Poetry's package dependencies
perceval-mozilla 0.3.4 - (2022-10-31)
* Update Poetry's package dependencies
perceval-mozilla 0.3.1 - (2022-06-24)
**Others:**
* Update GrimoireLab dependencies\
Dependencies have been updated to use the latest versions of
GrimoireLab packages:
- grimoirelab-toolkit >= 0.3
- perceval >= 0.19
perceval-mozilla 0.3.0 - (2022-06-02)
**New features:**
* Dependencies file for development\
Include a new requirements.txt file to install the development version
from the repositories.
perceval-mozilla 0.2 - (2018-05-21)
** New features improvements: **
* Backends updated to be compatible with Perceval 0.11.
* Creates backend added.
perceval-mozilla 0.1 - (2017-01-19)
** New features and improvements: **
* New set of backends added:
- Kitsune
- Mozilla Club
- ReMo
* Backends updated to be compatible with Perceval 0.5, using the namespace
schema and the new implementation of `BackendCommand`.
** Bugs fixed: **
* Offset was not stored in ReMo cache. This caused errors when trying
to resume with items from the cache. (1)