Pioreactor

Enhancements
- Initial support for RPi5! To use an RPi5, we recommend not upgrading the software, but using a fresh image install. Under the hood:
- we are using a new route to load the firmware on the HATs RP2040 (using `linuxgpio`)
- the hardware PWMs on the RPi5 use a different chip location. This required a new `rpi_hardware_pwm` release.
- new ENV variable, `HAT_PRESENT=1`, can be set to skip `is_HAT_present` checks.
- added the RPis unique MAC addresses to the `Manage -> System` tab on the Pioreactors page.
- added table `ir_led_intensities` to be able to be exported on the Exports page.
- added a new `smoothing_penalizer` config option to `[od_config]`. This parameter, which has default value 700, controls how much smoothing to apply to optical density measurements. This smoothing has always been applied, but now it's a config option.
- Cleaned up some UI interactions

Breaking changes
- `PWM` class is no longer initialized with a `duty_cycle`, instead:
- `PWM` class must be started with `start(initial_duty_cyle)`
- moved `get_rpi_machine` to `pioreactor.version`

Bug fixes
- Ack! I reintroduced a UI export bug. Fix is present going forward. For existing users, try the following: https://forum.pioreactor.com/t/new-pioreactor-release-23-11-18/179/2
- fixed a bug where stirring DC would jump up too high when RPM measured 0.

Enhancements

- Improvements to OD calibration and pump calibrations. Both now have a `-f` option to provide a json file with calibration data, to skip rerunning data-gathering routines. For example: `pio run pump_calibration -f pump_data.json`.
- Ability to update via our release_archives (available on the [Github release page](https://github.com/Pioreactor/pioreactor/releases)) via the UI. To turn this feature off (which is a recommended practice when you expose your UI publicly), add an empty file called `DISALLOW_UI_UPLOADS` to the `~/.pioreactor` directory.
- A new config option to change the max volume to dose when a larger dose volume is split. For example, if your chemostat asks to dose 1.6 ml, our internal algorithm will dose 0.75, 0.75 and 0.1 (this is to avoid overflow). The 0.75 was previously hardcoded, but is now a config `max_subdose` under section `[dosing_automation.config]` (default is still 0.75 ml).

Breaking changes

- Changes to `types.DosingProgram`, now it requires an MQTT client. Usually this is `automation.pub_client`. This is to avoid a memory leak!

Bug fixes

- Fixed an problem where an automation would not successfully end due to it being "blocked" by a `while` loop in the `execute`.
- Fixed a memory leak in dosing control when the automation would pump many many times.

- fix for exporting data from the UI

Breaking changes
- Merged the turbidostat automations into one. You can either select to target nOD or target OD, but not both!
- `ws_url` in the configuration now requires a fully qualified url. Example: `ws://...` or `wss://...`.
- Removed `morbidostat` dosing automation, users should try to use pid_morbidostat. The morbidostat code is still available to be added as a custom plugin here: https://github.com/Pioreactor/automation-examples/blob/main/dosing/morbidostat.py
- Removed `constant_duty_cycle` temperature automation. Again, the code is available here: https://github.com/Pioreactor/automation-examples/blob/main/temperature/constant_duty_cycle.py
- `pid_morbidostat` now explicitly uses the keyword arg `target_normalized_od`, instead of `target_od`. It always has been nOD.

Enhancements
- Both "Target OD" and "Target nOD" displayed and editable in the UI.
- Previously, if the LED channel was locked (most common when OD reading was running), then any changes to the LED intensity (via the UI) would be silently rejected. This is changed: we have added retry logic that will attempt to keep changing it a few more times (hopefully to avoid the lock)
- Added some light form validation in the automations dialog in the UI.
- New environment variable to skip loading plugins, `SKIP_PLUGINS`. Useful for debugging. Ex:

SKIP_PLUGINS=1 pio run stirring

- elements in the `field` array in automation yamls now can include a `type` option (`numeric` or `string` for now). Default is `numeric` if not specified.

Bug Fixes
- Fix experiment profile validation error
- The "Stop" button is always available now in the "Dosing" tab for "Manage all Pioreactors".
- Fix for Ngrok remote access.
- Fixed a race condition between starting an automation and not getting OD data in time.
- The automation form in the UI for pid_morbidostat was missing `volume`, that's been added now.

- No more waiting around for growth-rate-calculating to get to "Ready" state
- The "Label" step in the New Experiment flow is skipped if there is only 1 active Pioreactor in the cluster.
- Silenced the "LED is locked" warning - now it's a DEBUG level message.
- Fixed bug that wasn't passing lists correctly in `TopicToParserToTable`
- Faster boot times.
- Faster UI load times by gzip-ing assets.
- Fixed a bug where a plugin would not be loaded if it's name collided with a module in the stdlib. For example, putting `test.py` in `.pioreactor/plugins` would not be loaded, since when we tried to import `test`, it would load the stdlib's `test`, not the local plugin. This has been fixed.
- Simplify some UI elements.
- Security improvements.
- Reduce the default LED intensity in `light_dark_cycle` from 50% to 5%. This is more appropriate for sparse cultures.
- Fixed a race condition when starting a hotspot with boot config.ini.
- changed how `is_HAT_present` determine is the HAT is on the Pi. Previously, it used the i2c bus to check if the RP2040's firmware was active. This would fail if the HAT was present, but the firmware or i2c wasn't working. Now we check the EEPROM, which is a much more robust test.

- fix bug in `timeout` in `Stirrer.block_until_rpm_is_close_to_target` that wasn't using time correctly.
- Workers can now also be the local-access-point (aka the "router" in a network). Previously only leaders could.
- Experiment profiles now support a `log` directive with some dynamic templating:

- type: log
hours_elapsed: 0.025
options:
message: "{job} increasing to 800 RPM" alerts the message: "stirring increasing to 800 RPM"

See full example [here](https://github.com/Pioreactor/experiment_profile_examples/blob/main/08_logging.yaml).
- Experiment profiles now supports changing LEDs like any other job (i.e. they can use `start`, `update` and `stop` directives). See example [here](https://github.com/Pioreactor/experiment_profile_examples/blob/main/03_dosing_and_leds.yaml).
- Experiment profile clean up. I think there are less bugs!
- `pio clear_cache` now has an `as_int` option to look for ints - useful when clearing caches with ints as keys.
- fix issue where if an extra config.ini was provided in the /boot dir before a worker startup, adding the worker to a cluster would fail due to a permission issue.
- potential fix for RPi 3B and RPi Zeros not connecting to hotspots: change the `proto` to `wpa` in your config.ini on the Pioreactor with the local-access-point, and restart that Pioreactor. You config.ini should look like:


[local_access_point]
ssid=pioreactor
passphrase=raspberry
proto=wpa


**Changing to WPA does weaken the security however! It's easier for unwanted users to get onto this wifi.**

A more robust solution is in the works for RPi 3B and Zeros.

- Slight change to the API initialization of ADCReader. Take a look if you were using that class.