Nemo

Update notes
* The new adjustment request feature needs to be enabled in Customization -> User request and at least one user with the facility manager role needs to exist (for approval)
* A email template for the new adjustment request feature can be found in [the emails folder](https://github.com/usnistgov/NEMO/blob/master/resources/emails/adjustment_request_notification_email.html)

New features
* Added Tool qualification groups allowing to qualify a user on a group of tools at the same time. The feature is currently available in the Qualification and Training pages.
* Merged staff charges and remote work menu items. Both can now be found under `Administration -> Remote work` now
* Big expansion of the REST API now allows creating new users, accounts, projects etc. Actions available in the REST API are Creation, Update, Partial Update and Deletion. Thanks `MIT.nano` for the contribution!
* Added adjustment requests:
* If enabled in Customization, users can request adjustments from their usage page or from the global requests link
* The type of charges allowed is customizable, currently tool usage, area access and missed reservations are available
* Time limit for requesting adjustments is configurable and defaults to 2 weeks
* PIs can request adjustments for any eligible charges on their projects
* Adjustment requests can be exported in detailed administration and request page.
* Adjustments can only be reviewed by facility managers.
* If an adjustment is accepted, the user will receive confirmation, and a separate email will be sent to the User Office including optional instructions.
* If an adjustment is rejected, the user will receive notification including optional feedback.
* NOTE: Adjustment requests do not actually change charges. They are simply meant to allow users to create requests directly in NEMO, and adjustments should be made outside of NEMO in the facility's own billing system.

Improvements
* Now displaying the creator of an access request when editing/reviewing it
* Added stack trace for errors thrown during plugin loading
* Discipline is now only used for projects and was renamed "Project discipline"
* Including utility packages (vim, less) in the docker container
* Major update of the Stanford interlock implementation with error checks for returned values. Many thanks to `Dave Botsch` from `Cornell NanoScale Facility` for the suggestion and all the help debugging!
* Navigation bar will now collapse to the "hamburger" menu (similar to mobile experience) on medium sized screens
* Improved the notification model to allow for multiple notifications of the same type.
* Now allowing html in request description field (buddy requests, access requests etc.)
* Added default value for post usage questions
* Added customization to hide tool usage data history for regular users
* Staff availability can now be hidden from staff status
* The `Application identifier` display name for projects can now be customized (to PO, Project Id, etc.)
* Natural keys have been added for easier import/export of projects (by name), users (by username), accounts (by name) and any category type model (by name)
* Added Customization to make location and phone number not required when creating tools. Tool usage data history tab can also be hidden from regular users.
* Moved all policy rules into its own class
* Regular users can now be given detailed admin permissions and the will have access to the administration if applicable
* In detailed administration, locking/unlocking interlocks will now display errors
* Added access requests export button (both in detailed admin and regular page)
* Added new "checkbox" type post usage question
* Added a way to customize how the current setting for configuration is displayed
* Added remote work customizations, with options to automatically charge area access when working on remote project, to explicitly ask if work on behalf of a user should be considered remote, and to enable/disable remote charges validation
* Added a customization option to allow PIs to add/remove existing users to their projects
* Added a CSV parser for REST API as well as a file import option

Bug fixes
* Fixed project discipline not showing in project detail admin form
* Fixed 135 and more (broken links telling the admin to add model instances in detailed admin but django admin app is not installed)
* Fixed 134. Thanks r-xyz for the fix!
* Fixed User office staff not being able to see access requests
* Fixed infinite loop in authentication middleware when user doesn't have an account in NEMO.
* Fixed recurring outage across midnight not being created correctly
* Fixed issue when reading multiple sensors at once (connection not properly closed)

Libraries
* Django 3.2.16 -> 3.2.18 (vulnerability)
* cryptography 39.0.0 -> 40.0.1
* drf-excel 2.2.0 -> 2.3.0
* drf-flex-fields 1.0.0 -> 1.0.2
* pytz 2022.7.1 -> 2023.3
* django-filter 22.1 -> 23.1
* Pillow 9.4.0 -> 9.5.0
* pymodbus 3.1.1 -> 3.2.2

- fixed modbus issue with new version of pymodbus

Update notes
Support for Python 3.7 has been dropped. Python 3.8 or newer is now required:
* If you are using the docker image, it will be automatically taken care of.
* If you are installing NEMO using `pip`, you will need to update your version of python, as well as your start/stop services to point to gunicorn from python 3.8

New features
* Added **Safety** as a new main menu item. `Safety issues` and `Safety data sheets` have been moved under this new section to provide a main Safety "hub" in NEMO.
* Added `Safety items`:
* It is a customizable list of Safety bullet points, consisting of a name, a description and (optionally) a list of documents.
* Safety items can be grouped in Safety categories.
* PDF and Video documents (.mp4, .ogv, .webm, .3gp) can be opened directly in NEMO (popup)
* All other types of documents will take the users to the link/download the file.
* Users can search through Safety items name and descriptions.
* In `Customization -> Safety`, the Safety menu item can be hidden, as well as the Safety suggestions and SDS. Safety categories can be organized as flat/collapsible items all in one page, or as separate side sub-menu items.

Improvements
* Added update of os packages when building NEMO docker images, to guarantee the latest security patches and updates are installed.
* Allowing user office members to force a user out of a tool.

Bug fixes
* Fixed an issue on Chrome (Windows only) and Safari when clicking the back button after a spinner was shown would keep that spinner on indefinitely.

Libraries
* gunicorn 19.9.0 -> 20.1.0
* cryptography 38.0.4 -> 39.0.0
* Pillow 9.3.0 -> 9.4.0
* pymodbus 2.5.3 -> 3.1.1
* pytz 2022.6 -> 2022.7.1
* requests 2.28.1 -> 2.28.2

Bug fix
- fixed training not being recorded when click the confirm button
- fixed non technical staff not having access to staff status view options and past/future calendar
- fixed user office needing access to areas when "allow_staff_access" is set on physical access levels
- fixed user office not recognized as staff in occupancy screens

Bug fix
- fixed csrf token issue on tool qualification customization

Update notes

1. New `User Office` and `Accounting Officer` roles have been added. They allow a greater level of granularity in NEMO. Consequently, `Staff` users won't be able to edit user information or see the billing information for other users by default. To give a user any of the new roles, go to `Detailed Administration -> Users` select the user and **check** the `User office` or `Accounting officer` checkbox and **uncheck** the `Staff` box. More information can be found in the feature manual.
1. A cron job for access expiration email reminders needs to be enabled for the feature to work. A systemd version for docker is available in the [systemd folder](https://github.com/usnistgov/NEMO/tree/master/resources/systemd).
2. A cron job for managing tool qualifications needs to be enabled for the feature to work. A systemd version for docker is available in the [systemd folder](https://github.com/usnistgov/NEMO/tree/master/resources/systemd).
3. A cron job for managing recurring charges needs to be enabled for the feature to work. A systemd version for docker is available in the [systemd folder](https://github.com/usnistgov/NEMO/tree/master/resources/systemd).


New features
* Added recurring consumable charges, which allows charging users for the same consumable at a given frequency. For example, charging user for renting user bins every year.
* Recurring charges can be exported (including potential errors)
* The quantity can be forced to a certain number in customization (for example when it should only and always be just one)
* The recurring charges can be locked so non facility managers can only assign them to user and not change the frequency or linked consumable.
* A consumable category can be set in customization to limit recurring charges to only consumables belonging to that category.
* Validating customers (inactive, access expired, etc.) can be deactivated for recurring charges in customizations.
* Users can set email reminders for their recurring charges in User preferences.
* Added Tool qualification expiration to remove tool qualification from users after a certain number of days. It needs to be enabled in Customization -> User. An example of the email template can be found [here](https://github.com/usnistgov/NEMO/blob/master/resources/emails/tool_qualification_expiration_email.html). There are 2 separate cases that can be customized:
1. Number of days since the user last used a tool. For example, remove tools from the user qualifications if they have not used it for 6 months.
2. Number of days without using a tool since the user was trained on that tool. For example, remove tools from the user qualifications if they haven't used it in the 2 weeks after being trained on it.
* Added Access expiration reminder email to remind users a certain number of days before their access expires. It needs to be enabled in Customization -> User. An example of the email template can be found [here](https://github.com/usnistgov/NEMO/blob/master/resources/emails/user_access_expiration_reminder_email.html).
* Added an optional `EMAIL_USE_DEFAULT_AND_REPLY_TO` option in `settings.py` to use the default server email for all communication and setting the reply-to of the email to the actual sender. This option is helpful when using an email server that doesn't allow spoofing (for example a unique Gmail address).
* Added an optional `USERNAME_REGEX` option in `settings.py` to validate usernames.
* Added an optional `MAIN_URL` option in `settings.py`. This is useful when running multiple instances of NEMO, so all email links are sent to the same URL.
* Added audit log library that can be customized to track any changes in NEMO. See [setup instructions on the wiki](https://github.com/usnistgov/NEMO/wiki/Audit-log-and-tracking-changes).
* Added a few contributions from `Cornell NanoScale Facility`:
* Added `Discipline`, a new configurable category that can be set on projects (Chemistry, Electronics etc.).
* Added `Safety trainings`, a new configurable list of trainings that can be checked/unchecked for each user.
* Added `Onboarding phases`, a new configurable list of items that can be checked/unchecked for each user.
* Added user and project document upload, which can be enabled in `Customization -> Application` and `Customization -> Projects & accounts`.

Improvements
* Added `unit_id` for interlocks using Modbus and added last reply time.
* Added tooltips with tool information in the status dashboard page.
* All email templates can now use global variables like `site_title`, `facility_name` etc.
* Updated autocomplete to be either synchronous or asynchronous, the ladder considerably speeding load time of pages like "Users".
* The user search bar is now available when viewing/modifying users to facilitate switching between them.
* Consumables can now be `reusable` which will prevent the quantity from ever decreasing when withdrawals are made.
* Added customization in `Customization -> User` to hide inactive user in the users page and made the modify user page go back to previous pagination page upon success. Thanks `USC Nanofab` for the contributions!
* Added customization in `Customization -> Project & accounts` to hide inactive accounts, inactive projects, and to collapse the project list by default. Thanks `Polytechnique Montréal` for the contributions!
* Added a way to change the calendar time format in `Customization -> Calendar`.
* Updated buttons in the entire application to have a consistent color and icon for same functionality.
* Optimized Safety Data Sheets page and added sorting by Hazard.
* Users now have the option to opt out from some of the email notifications in User preferences.
* Added Safety Data Sheets CSV export.
* Made pagination number of results per page sticky when navigating away and back. Also added an "all" option.
* Broadcast email feature now allows selecting multiple tools/areas/projects/accounts.

Bug fixes
* Fixed delayed docker container removal when stopping NEMO. Thanks r-xyz for the contribution!

Librairies
* added django-audit-log 2.2.1
* added pytz 2022.6
* cryptography 37.0.4 -> 38.0.4
* django 3.2.15 -> 3.2.16 (vulnerability)
* django-auditlog 2.2.0 -> 2.2.1
* django-mptt 0.13.4 -> 0.14.0
* djangorestframework 3.13.1 -> 3.14.0
* drf-excel 2.1.0 -> 2.2.0
* Pillow 9.2.0 -> 9.3.0