Socs

Description
v0.9.0 introduces some new tools and features, such as the ocs-client-cli tool and param decorator. Test coverage is also greatly expanded. One change in the API that requires attention is the renaming of `MatchedClient` to `OCSClient`. Users should update their scripts where `MatchedClient` is used, as it will be removed in a later release. Full change log below.

Change Log
New Features
* Introduce new Client CLI tool (218)
* Create new decorator for tasks/processes that accept parameters (223)
* Expand test coverage with new unit and integration tests (230 and 231)
* Rename MatchedClient to OCSClient (236)

Documentation Updates
* Some bug fixes and a bunch of OpSession documentation work (220)
* Establish Agent documentation guidelines (221)
* Add sphinx extension that auto-links to source code (225)
* Create rtd config file, split requirements, and pin versions (235)
* Update Client Documentation (236)

Bug Fixes
* Check if args are passed to MatchedClient (206)
* Deal gracefully with crossbar not being found (208)
* Shutdown AggregatorAgent on PermissionError (213)
* Protect test imports (217)
* Some bug fixes and a bunch of OpSession documentation work (220)
* Only perform check on param if value is given (228)
* Update feed message data check to handle bools (229)
* Core updates, prelude for major ocsbow (241)

Misc.
* Reorganize Dockerfile to leverage build cache (212)
* Add tests for ocs_agent.py (227)
* Core updates, prelude for major ocsbow (241)
* Build and Deploy to PyPI (243)
* Prepare for v0.9.0 release (244)

Description
v0.8.0 includes some bug fixes and a few new features, including feed exclusion, for handling high sample rate feeds separately from InfluxDB, and Operations Monitoring, for keeping an eye on OCS task/process state. We also update to Ubuntu 20.04 within the OCS Docker containers. This update was required to fix the build process. Users who are still on v0.7.1 who run into container build problems (say if you're developing a new agent) should update to v0.8.0. Full change log below.

Change Log
New Features
* Add feed exclusion options, allowing feeds to be published exclusively to influxdb or the HK aggregator (187)
* Add operations monitoring, providing a way to monitor task/process status via Grafana (188)
* Update to so3g v0.1.0-24-g5645096 (190)
* Add support for InfluxDB "line" protocol and gzip to InfluxDB Publisher, improving write performance (191)
* Switch ocs-util to jupyterlab (201)

Documentation Updates
* Add docs about logging within Agents (195)
* Clarify the permissions structure for ocs-util (202)

Bug Fixes
* Catch InfluxDBServerErrors when the InfluxDB server is down (191)
* Fix coverage reporting (192)
* Fix InfluxDB Publisher String Handling (198)

Misc.
* Update to Ubuntu 20.04 based Docker images (193)
* Drop wampy module as a dependency (194)
* Drop ipywidgets and ocs_widgets (196)

Description
v0.7.1 is mostly bug fixes, however there are some new important features. OCS Web has been expanded to include a new control panel system for Agents, allowing tasks and processes to be started via the web interface for supported Agents (163). Agents should now be robust against short term disconnection from the crossbar server (180). This means they will stay up during a crossbar reboot if one is required. Full change log below.

Change Log
New Features
* Add ability to skip file check when using g32influxdb (https://github.com/simonsobs/ocs/pull/162)
* Add OCSWeb control panel system for Agents (https://github.com/simonsobs/ocs/pull/163)
* Create ocs-crossbar image to replace sisock-crossbar image (https://github.com/simonsobs/ocs/pull/166)
* Base new ocs-crossbar image off of official crossbar.io provided image (184)
* Make OCSAgents robust against short duration crossbar dropouts (180)
* Create Pacemaker class for regularizing sampling rates in Agents (178)

Documentation Updates
* Document and establish Loki as log aggregation tool for OCS (https://github.com/simonsobs/ocs/pull/161)
* Improve documentation builds with mock imports (165)
* Expand session.data documentation (170)

Bug Fixes
* Small g32influxdb bug fix (https://github.com/simonsobs/ocs/pull/162)
* Allow str type on OCS Feed (164)
* site_config: use a string comparison instead of identification (169)
* Use 'dumb-init' system within containers to start Agents (https://github.com/simonsobs/ocs/pull/171)
* Add default `agent_class` attribute in agent instantiation (177)
* Remove empty blocks' block_name from frame (179)
* Ignore and log invalid kwargs passed to Providers (183)

Misc.
* Remove crossbar from main requirements.txt file (173)
* Remove all references to 'prefix' (182)

v0.7.0 notably introduces so3g HK v2 to OCS, as well as field name validation to OCS Feeds/the HK Aggregator. Full changelog below.

Changelog
* Update the HK Aggregator to so3g HK v2 (updating to so3g v0.1.0) (157, 128)
* Enforce valid field names in OCS feeds and in the HK Aggregator (132)
* Create `g32influx` script, for uploading data from .g3 files to InfluxDB (153)
* Add sotodlib to ocs-util container (4a239703b1c6a36b717c94e687156df92ab4c9d8)
* Add `fresh_time` to feed registration arguments (85)
* Improve the aggregator's `session.data` (85)
* make MatchedTask and MatchedProcess callable (117)
* Add invalid field name check to `checkdata` script (147)
* Improve setup.py (151)
* Improve OCS Feed and HK Aggregator documentation (137)
* Various bug fixes (2473e4c32c14509b92100de9982f2f921c864ec3, c2cab184edc6a9a28e9e6e5927b6bb37a2b351b1, 123)
* Clean up some old files, i.e. `ocs.cfg`, old clients
* Migrate repository CI Pipeline to GitHub Actions (146, 158)
* Establish contributing guidelines (146)

It's been ~9 months or so since we updated the tag, so it's about time for a new release. Here's a summary of what's been added since we last updated:

- FakeDataAgent has been promoted to a core OCS Agent!
- We restructured some docker internals
- Rebuilt the aggregator and registry agents to be more resilient
- Created the OCS web monitor, which provides a view of the ocs system from your web browser
- Added smarter argument parsing and site-config parsing
- Added InfluxDB backend for passing data to grafana
- Overhauled documentation
- Added ocs-util docker

What's Changed
v0.5.2 mostly consists of minor new features and many bug fixes. We have also updated the build backend, resulting in slight changes in the final source distribution. The `tests/` directory is no longer included. Those looking to run the tests should get them directly from this repository.

The base ocs docker image has been updated to remove several redundant base layers in 714. See https://github.com/simonsobs/ocs/pull/397 for some of the details. This should result in smaller socs images. If you care about the structure of the image, take note that `so3g` is now pip installed instead of being compiled in lower down base layers.

Some breaking changes -- the `ACTiCameraAgent` was renamed to `HTTPCameraAgent` to reflect broader applicability to more models of webcam. See [the agent docs](https://socs.readthedocs.io/en/main/agents/http_camera.html) for updated configuration examples. Also, the 'fake_data' argument in the Lakeshore372 Agent that would generate random data for testing as been removed. If you're looking to test your ocs/Grafana setup, use the [ocs `FakeDataAgent`](https://ocs.readthedocs.io/en/main/agents/fake_data.html).

New Features
* Update ACTiCameraAgent to HTTPCameraAgent to include other HTTP camera brands by davidvng in https://github.com/simonsobs/socs/pull/700
* Added sample heater display mode check in custom_pid function. by mjrand in https://github.com/simonsobs/socs/pull/729
* Adds grip and ungrip fns to hwp-supervisor with safety checks by jlashner in https://github.com/simonsobs/socs/pull/708
* Scpi psu reconnect by agthomas-uc in https://github.com/simonsobs/socs/pull/726
* Adding support for direct Ethernet SCPI PSU Agent by CAWNoodle in https://github.com/simonsobs/socs/pull/715
* Adds extra db connection options to suprsync agent by jlashner in https://github.com/simonsobs/socs/pull/748
* HWP Encoder Update (Restart process) by bbixler500 in https://github.com/simonsobs/socs/pull/738
* HWP Gripper Updates (Alarm group and restart processes) by bbixler500 in https://github.com/simonsobs/socs/pull/737
* Add error handling to Cryomech PTC drivers and agent acq process by BrianJKoopman in https://github.com/simonsobs/socs/pull/767
* ACU: more care when setting up go_to by mhasself in https://github.com/simonsobs/socs/pull/768
* ACU: acuconfigs.yaml setting for forcing sequential moves by mhasself in https://github.com/simonsobs/socs/pull/778

Bug Fixes
* added handling of pid error response by bbixler500 in https://github.com/simonsobs/socs/pull/686
* hw_pid: delete initial set_direction by ykyohei in https://github.com/simonsobs/socs/pull/703
* HWP supervisor agent brake bugfix by bbixler500 in https://github.com/simonsobs/socs/pull/720
* HWP supervisor iboot indexing bugfix by bbixler500 in https://github.com/simonsobs/socs/pull/722
* Publish BF logs only if data extracted from them by BrianJKoopman in https://github.com/simonsobs/socs/pull/530
* HTTPCameraAgent Handle token lease and usage by davidvng in https://github.com/simonsobs/socs/pull/731
* Various HTTPCameraAgent Fixes by davidvng in https://github.com/simonsobs/socs/pull/733
* pysmurf-monitor main process robustness by jlashner in https://github.com/simonsobs/socs/pull/732
* HTTPCameraAgent write file change by davidvng in https://github.com/simonsobs/socs/pull/740
* HTTPCameraAgent Add more exceptions by davidvng in https://github.com/simonsobs/socs/pull/746
* Fix 2 ACU bugs by mhasself in https://github.com/simonsobs/socs/pull/747
* Ls372 custom pid update by mjrand in https://github.com/simonsobs/socs/pull/753
* Inconsequential correction to IFM SBN246 temperature decode. by swh76 in https://github.com/simonsobs/socs/pull/761
* Add optional encoding to moxa `Serial_TCPServer` by d-hoshino2626 in https://github.com/simonsobs/socs/pull/764
* hwp-pid direction read bugfix by bbixler500 in https://github.com/simonsobs/socs/pull/728
* Fix typo in pysmurf-controller relock function by jlashner in https://github.com/simonsobs/socs/pull/783

Documentation
* Remove unused --mode arg from example config by BrianJKoopman in https://github.com/simonsobs/socs/pull/719
* Fix typo in the docstrings for bias commands by BrianJKoopman in https://github.com/simonsobs/socs/pull/777

Version Updates
* [pre-commit.ci] pre-commit autoupdate by pre-commit-ci in https://github.com/simonsobs/socs/pull/713
* Bump simonsobs/ocs from v0.11.1-1-gb6cbd88 to v0.11.1-9-gfe03421 by dependabot in https://github.com/simonsobs/socs/pull/741
* Bump simonsobs/ocs from v0.11.1-9-gfe03421 to v0.11.2-8-gfa333dd by dependabot in https://github.com/simonsobs/socs/pull/751
* Bump simonsobs/ocs from v0.11.2-8-gfa333dd to v0.11.3-3-g63fd5ef by dependabot in https://github.com/simonsobs/socs/pull/757
* [pre-commit.ci] pre-commit autoupdate by pre-commit-ci in https://github.com/simonsobs/socs/pull/773

Packaging
* Update ocs base image to use newly reworked image by BrianJKoopman in https://github.com/simonsobs/socs/pull/714
* Pin pysnmp version to 4.4.12 by davidvng in https://github.com/simonsobs/socs/pull/718
* Switch build-backend from setuptools to hatchling by BrianJKoopman in https://github.com/simonsobs/socs/pull/758
* Bug fix of wiregrid_actuator/Dockerfile for Ubuntu update by sadachi5 in https://github.com/simonsobs/socs/pull/759

Misc.
* Set timeout for pytest workflow to 1 hour by BrianJKoopman in https://github.com/simonsobs/socs/pull/742
* cryomech: Move PTC class to drivers module by BrianJKoopman in https://github.com/simonsobs/socs/pull/771
* Remove 'fake_data' arg for random data generation by BrianJKoopman in https://github.com/simonsobs/socs/pull/776

New Contributors
* agthomas-uc made their first contribution in https://github.com/simonsobs/socs/pull/726
* CAWNoodle made their first contribution in https://github.com/simonsobs/socs/pull/715
* swh76 made their first contribution in https://github.com/simonsobs/socs/pull/761

**Full Changelog**: https://github.com/simonsobs/socs/compare/v0.5.1...v0.5.2