Udata

New & Improved

- Topics have been refactored and are no more deprecated:
- Topics creation, update and deletion are now opened to all users [2898](https://github.com/opendatateam/udata/pull/2898)
- Topics are now `db.Owned` and searchable by `id` in dataset search [2901](https://github.com/opendatateam/udata/pull/2901) [#2917](https://github.com/opendatateam/udata/pull/2917)
- Remove `deleted` api field that does not exist [2903](https://github.com/opendatateam/udata/pull/2903)
- Add `created_at`field to topic's model [2904](https://github.com/opendatateam/udata/pull/2904)
- Topics can now be filtered by `tag` field [2904](https://github.com/opendatateam/udata/pull/2904)
- Topics can now be queried by test search in `name` field with `q` argument [2904](https://github.com/opendatateam/udata/pull/2904)
- Add support for a CSW harvester using DCAT format [2800](https://github.com/opendatateam/udata/pull/2800)
- Add German to udata translations [2899](https://github.com/opendatateam/udata/pull/2899)[2909](https://github.com/opendatateam/udata/pull/2909)
- Add harvesters count to site metrics [2890](https://github.com/opendatateam/udata/pull/2890)
- Use a single session for reindex [2891](https://github.com/opendatateam/udata/pull/2891)

- Fix site title and keywords never get updated [2900](https://github.com/opendatateam/udata/pull/2900)
- Reuse's extras are now exposed by API [2905](https://github.com/opendatateam/udata/pull/2905)
- Add configuration settings to enhance cookies security [2910](https://github.com/opendatateam/udata/pull/2910)
- Add items.dataset to HarvestJob indexes [2907](https://github.com/opendatateam/udata/pull/2907)
- Consider acronym when suggesting organization [2918](https://github.com/opendatateam/udata/pull/2918)

Fixes

- Return 400 instead of 500 in case of not ObjectID arg in API [2889](https://github.com/opendatateam/udata/pull/2889)
- Fix default community resource sort parser [2908](https://github.com/opendatateam/udata/pull/2908)

Deprecation

- Python 3.7 is now deprecated and will be removed in upcoming release [2859](https://github.com/opendatateam/udata/pull/2859)
- GeoZone model will be heavily refactored for a simplified version [2878](https://github.com/opendatateam/udata/pull/2878)

- Fix slug overflow with index suffix when reaching max_length [2874](https://github.com/opendatateam/udata/pull/2874)
- Add extra field to topic model and add it to the dataset search adapter [2876](https://github.com/opendatateam/udata/pull/2876)
- Upgrade pyyaml in develop and doc deps [2880](https://github.com/opendatateam/udata/pull/2880)
- Expose dataset's `*_internal` dates in a nested `internal` nested field in api marshalling [2862](https://github.com/opendatateam/udata/pull/2862)
- Add `business_number_id` metadata for organizations [2871](https://github.com/opendatateam/udata/pull/2871) [#2887](https://github.com/opendatateam/udata/pull/2887)
- Return 403 when posting comment on discussion closed [2881](https://github.com/opendatateam/udata/pull/2881)
- Ensure rdf parsed frequency is lowercase [2883](https://github.com/opendatateam/udata/pull/2883)
- Add a dict of URIs to replace in a RDF graph at harvest time [2884](https://github.com/opendatateam/udata/pull/2884)
- Fix duplicate recipients in new comments mail [2886](https://github.com/opendatateam/udata/pull/2886)
- Add type to resource csv adapter [2888](https://github.com/opendatateam/udata/pull/2888)

- Improve DCAT harvest of mime type [2857](https://github.com/opendatateam/udata/pull/2857)
- Don't crash on files not found when purging resources [2858](https://github.com/opendatateam/udata/pull/2858)
- Improve DCAT catalog exposed [2860](https://github.com/opendatateam/udata/pull/2860)
- Use the resource's extra `analysis:last-modified-at` in the `last_modified` property [2863](https://github.com/opendatateam/udata/pull/2863)
- Add optionnal harvest validation form [2864](https://github.com/opendatateam/udata/pull/2864)
- Fix dataset list default sorting [2867](https://github.com/opendatateam/udata/pull/2867)
- Update API doc link [2866](https://github.com/opendatateam/udata/pull/2866)
- Update admin quality progress bar [2872](https://github.com/opendatateam/udata/pull/2872)

- Specify *public* datasets and reuses in admin count [2852](https://github.com/opendatateam/udata/pull/2852)
- Fix url params being stripped in markdown for internal URLs [2855](https://github.com/opendatateam/udata/pull/2855)

- Upgrade development dependencies [2844](https://github.com/opendatateam/udata/pull/2844)
- Compile translations for testing [2845](https://github.com/opendatateam/udata/pull/2845)
- Add user arg to discussion list API [2842](https://github.com/opendatateam/udata/pull/2842)
- No more sending email, slug and user name to sentry [2846](https://github.com/opendatateam/udata/pull/2846)
- Add test for passwordless user [2848](https://github.com/opendatateam/udata/pull/2848)
- Parse IANA and EUROPA URIs format [2849](https://github.com/opendatateam/udata/pull/2849)
- Dataset last update sorting:
- Transmit dataset's `last_update` field to search service [2847](https://github.com/opendatateam/udata/pull/2847)
- Check if dataset's and resource's harvesting date `modified_at` are ealier than today's date [2850](https://github.com/opendatateam/udata/pull/2850)
- Upgrade mongoengine and storage deps [2839](https://github.com/opendatateam/udata/pull/2839):
- Upgrade flask-storage 1.0.0 -> 1.3.2
- Upgrade flask-mongoengine 0.9.5 -> 1.0.0, now returning a ValidationError on get_or_404 on invalid id
- Upgrade mongoengine 0.26.0 -> 0.27.0
- Prevent raising unecessary error in index command [2851](https://github.com/opendatateam/udata/pull/2851)
- Use `datetime.utcnow` to make sure to handle utc datetimes [2853](https://github.com/opendatateam/udata/pull/2853)

- Fix XSS vulnerability: escape user content in selectize items in admin [2843](https://github.com/opendatateam/udata/pull/2843)
- Fix schema is undefined when checking for schema.url in admin resource form [2837](https://github.com/opendatateam/udata/pull/2837)
- Fix to_naive_datetime in harvest preview [2835](https://github.com/opendatateam/udata/pull/2835)
- :warning: Flask-Security update to enable `GenericResponses` [2826](https://github.com/opendatateam/udata/pull/2826):
- Upgrade Flask-Security 4.5.1 -> 5.1.1
- Upgrade WTForms 2.2.1 -> 3.0.1
- Upgrade WTForms-json 0.3.3 -> 0.3.5
- New security email template for existing users
- Fix SelectField validation failure following WTForms upgrade [2841](https://github.com/opendatateam/udata/pull/2841)
- Add `format_timedelta` to `udata.i18n` [2836](https://github.com/opendatateam/udata/pull/2836)
- Improve send_mail resilience with refused address among recipients [2840](https://github.com/opendatateam/udata/pull/2840)