Pioreactor

Enhancements

- `pio logs` now includes the UI logs (if run on leader).
- introduce a new od_reading config,`turn_off_leds_during_reading`, which enables / disables turning off the other LEDS during an OD snapshot. By default, it is set to 1 (enables).
- leader-only Pioreactors also have a `config_<hostname>.local` file now.
- a new top-level section in experiment profiles, `inputs`, allows you to define parameters that can be used in expressions. This is useful if you are copy the same constant over an over again, and want a quick way to change it once. Example:

yaml
inputs:
growth_phase_temp: 37.0
stationary_phase_temp: 30.0
od_threshold: 1.6

common:
jobs:
temperature_automation:
actions:
...
- type: update
hours_elapsed: 12.0
if: ${{ ::od_reading:od1.od < od_threshold }}
options:
target_temperature: ${{ stationary_phase_temp }}
- type: update
hours_elapsed: 12.0
if: ${{ ::od_reading:od1.od >= od_threshold }}
options:
target_temperature: ${{ growth_phase_temp }}



Bug fixes

- more resilience to "UI state" diverging from "bioreactor state". Often, this occurred when two jobs stared almost immediately (often a networking issue), and the last job would halt since it couldn't get the required resources, however any MQTT data would be overwritten by the last job. Now, multiple places in the request pipeline will reduce duplication and prevent two jobs from starting too close to each other.
- improved stirring clean up when stopped in quick succession after starting.
- if a network isn't found, the `monitor` job will not stall, but warn and continue.
- fixed HAT warning for HAT-less leaders.

Breaking changes

- the RP2040 firmware is now on i2c channel 0x2C (previously 0x30). This is to solve an annoying `i2cdetect` issue where the i2c channel would lock up.
- the web server now writes its logs to the same location as the app: `/var/log/pioreactor.log`. Those wishing to keep the old location can use a new configuration parameter `ui_log_file` to `[logging]` section and set it to `/var/log/pioreactorui.log`.
- removed `psutil` and `zeroconf` Python packages from new images. We replaced their functionality with built-in routines.
- in config.ini, the section `od_config` renamed to `od_reading.config`, and `stirring` is `stirring.config`. When you update, a script will run to automatically update these names in your config.inis.

Enhancements

- improvements to the UI's experiment profile preview.
- `hours_elapsed()` is a function in profile expressions, which returns the hours since the profile started.
- `unit()` can be used in mqtt fetch expressions. Example: `unit():stirring:target_rpm` is identical to `::stirring:target_rpm`. The latter can be seen as a shortened version of the former.
- experiment profiles can have a `description` in the `job` field (i.e. at the same level as `actions`).
- Updated Raspberry Pi OS image to 2024-07-04.
- Vendoring the TMP1075 library, which also fixes the RPi5 error.
- In places where the ipv4 is displayed (Inventory page, System tab, pio workers status, etc), *all* ipv4 addresses are displayed.

Breaking changes

- remove the temperature_control, dosing_control, and led_control abstractions. These were introduced early in the Pioreactor software as a way to quickly change automations, but they have been more of a wort than a win. While working on the internals of experiment profiles recently, it became more and more clear how poor this abstraction was. The removal of them has some consequences and some backward incompatibilities:

- updating experiment profiles: experiment profiles that have a `*_control` job will need to be updated to use `*_automation`, _eventually_. For now, we are allowing `*_control` in profiles: in the backend, we are renaming `*_control` to `*_automations`, but a warning will be produced. Later, we'll remove this renaming and profiles will need to be completely updated. Example:
yaml
experiment_profile_name: start_temp_control

metadata:
author: Cam DP

common:
jobs:
temperature_control:
actions:
- type: start
hours_elapsed: 0
options:
automation_name: thermostat
target_temperature: 30
- type: stop
hours_elapsed: 12
temperature_automation:
actions:
- type: update
hours_elapsed: 6
options:
target_temperature: 35


becomes:

yaml
experiment_profile_name: start_temp_control

metadata:
author: Cam DP

common:
jobs:
temperature_automation:
actions:
- type: start
hours_elapsed: 0
options:
automation_name: thermostat
target_temperature: 30
- type: stop
hours_elapsed: 12
- type: update
hours_elapsed: 6
options:
target_temperature: 35


- update plugins. For users using, specifically, the high-temp plugin, or temperature-expansion-kit plugin, new plugins will be released. Look on the forums, or documentation, for update instructions.

The benefits of removing this abstraction is much less code, less overhead, easier developer experience, and overall simplification. Later, we may create a new abstraction, but now we are moving abstractions back to level 0.

- `log` in experiment profiles now uses expressions instead of Python string formatting. For example: `The unit {unit} is running {job} in experiment {experiment}` should be replaced by expressions in the string: `The unit ${{unit()}} is running ${{job_name()}} in the experiment ${{experiment}}`. Note: `{job}` is now `${{job_name()}}`.
- `cycle_media` and `cycle_alt_media` now publish dosing events, and will be recorded by dosing automations, and the db.


Bug fixes

- When pausing temperature automations, the heater now turns off and stays off until unpaused. This is the intended behaviour.

Hotfix release for 24.7.3. This pins blinka to a specific version which does not install numpy.

Enhancements
- A new live preview in the UI's experiment profile editor. This preview tool is useful for getting immediate feedback when writing a profile. We'll keep on adding to this to improve the edit-profile workflow - please send us feedback!
- new `when` action type in experiment profiles that will execute an action (or list of actions) when some expression is true. For example, start a chemostat when a threshold OD is first achieved, log a message when event is triggered, or monitor a bioreactor parameter and execute an action if it goes out of bounds.
- New config `turbidostat.config` that can be used to modify some internal turbidostat settings:

[turbidostat.config]
signal_channel=2
od_smoothing_ema=0.5

- Better user interaction on the Pioreactors page when the assigned experiment and "viewing" experiment are different.
- Select / Deselect all Pioreactors to assign to an experiment faster.
- Added `unit()` function to experiment profiles expressions that returns the unit name the expression is evaluated for. Ex: `if: ${{ unit() == worker01 }}`.
- Added `job_name()` function to experiment profiles expressions that returns the job_name the expression is evaluated for. Ex: `if: ${{ job_name() == stirring }}`.
- Added `experiment()` function to experiment profiles expressions that returns the experiment the expression is evaluated for. Ex: `if: ${{ experiment() == exp001 }}`.

Breaking changes
- significant web backend API changes! See list of rules in docs.

Bug fixes
- Fix UI code editor from being unresponsive when all the text was removed.
- Experiment profiles won't be overwritten if providing the same filename as an existing profile.

Enhancements
- we changed the "auto" algorithm for picking a good `ir_led_intensity`. We now try to maximize the intensity, up to some constraints around saturating ADCs, LED longevity, and signal. In general, we expect a higher IR intensity, but this will help with noise and detecting lower signals.
- More improvements on the Pioreactor-specific page: added charts and a logs table.
- Added a "retry failed tests" to the UI's self-test dialog.
- `pio run self_test` has a new flag `--retry-failed` to only retry tests that failed in the previous run (if any).
- better clean up when a worker is removed from a cluster.
- reduce the mosquitto logs to reduce writes to disk and speed up connections.
- Use lexicographical ordering for all displays of workers in the UI
- **This only applies to new installed images, and not updates.** Updated to the latest RPI image, 2024-03-15, -> linux kernel update to 6.6. Recent versions of linux have improved support for usb wifi devices.
- **This only applies to new installed images, and not updates.** leader-only images will install worker Python libraries.
- **This only applies to new installed images, and not updates.** all experiment data will be deleted when the experiment is deleted.
- performance improvements

Breaking changes
- Changed the web backend API endpoints for time-series, logs, shutdown, reboot, and plugins to be more RESTful. See docs for updated rules in the docs.

Bug fixes
- fix performing an "undo" when editing the config.ini and experiment profiles.
- fix **Pioreactor v1.1** bug when change target temperature mid cycle causing the inferred temperature to change significantly.
- if a worker disconnected from the network, messages are queued in memory until the network reconnects. This has two problems. The first is that there is a finite amount of memory, and we don't want to OOM. The second is that when the worker(s) reconnect, there is a flurry of messages. For some jobs that use messages as events, this can cause multiple triggers in quick succession. We've added some logic that helps avoid these situations:
1. we max the queue of unsent messages to 100 (arbitrary)
2. in important jobs, like temperature automations, it will only respond to "recent" messages and not old messages.

Highlights
- New /pioreactor/`worker-name` page in the UI for a detailed view of an individual Pioreactor, including a realtime visualization of the Pioreactor!

Enhancements
- UI backend now supports external MQTT broker. This configuration lives in the same place as the existing MQTT settings: in the config.ini, under `[mqtt]`.
- Added groupings on the Experiment drop down to organize "Active" and "Inactive" experiments. An active experiment has >= 1 Pioreactor assigned to it.

Breaking changes
- New log topic that partitions by the level. This should make subscribers to the log topic slimmer (like the UI, who previously would have to accept _all_ messages and filter to what they needed). Should result in a performance increase.

Bug fixes
- Fix for Pioreactors page when _no workers are added to the cluster_.
- Fix for UI labels when trying to remove labels from Pioreactors.
- Improvements to REF self-tests.