Labbench

Changed
- Fixed platform portability problems in installing `labbench` cli

**API incompatible with labbench<=0.20 after major refactor**

Added
- Unit tests for lb.concurrently and lb.sequentially in test_concurrently.py
- `lb.NonScalar` trait type
- `lb.hide_in_traceback` decorator scrubs the decorated function from tracebacks
- `lb.Rack`, which also replaces `lb.Testbed`
- `lb.Coordinate`, which defines methods for `Rack` as a sequence of other functions with mixed threading
- `lb.HDFLogger` for output to an HDF file
- `lb.ShellBackend` replaces `lb.CommandLineWrapper`, and is defined by settings annotations

Changed
- Default values of Device value traits can now be set by passing keyword arguments when subclassing
- Show warnings on trait assignment typos like `device.frequency = 5` instead of `device.state.frequency = 5`
- `state` or `settings` traits can be defined directly in a `labbench.Device` class. Settings are defined as annotations ('`:`') and states are defined with assignment ('`=`')
- Add first 40 lines of CommandLineWrapper output to debug logs
- Removed logger warnings when calls to CommandLineWrapper.kill() do not kill any process
- Tightened the message about a pending exception in lb.concurrently
- The arguments and return value of sequentially have been corrected to match those in concurrently
- Testbed objects now support entering contexts of specified types first, which are listed (in order) by the new enter_first class attribute
- concurrently and sequentially now raise an exception of two callables have the same name; specify a different name with a keyword argument instead to avoid naming conflicts
- text file outputs in relational databases are now encoded as utf-8
- Removed Trait parameters `command`, `default_value`, `read_only`, and `write_only`; replaced with Trait parameters `key`, `default`, `settable`, `gets`, `allow`
- Removed Device methods `__get_state__`, `__set_state__`; added methods `get_key`, `set_key`
- Replaced Device methods `connect` and `disconnect` with `open` and `close` to more closely match python convention
- Support for updating default values of settings in subclasses as annotations
- Reduced import time by waiting to import heavier packages pyvisa and pandas
- lb.notebook is no longer pulled in by default; importing it now injects wrappers around builtins.range and np.linspace
- `host_log.txt` is now in YAML
- `CommandLineWrapper` is now `ShellBackend`
- Renamed the `logger` attribute to `_console` in Rack and Device to reduce the confusing overuse of the word "logger"
- `lb.BoundedNumber` (and subclasses `lb.Int`, `lb.Float`) now support creating derived Traits that act as arithmetic transformations, calibration against `device.setting`, and calibration against lookup tables
- `feather-format` is now an explicit dependency, because it is no longer (always?) pulled in by `pyarrow`
- Logger messages are only emitted after exceptions on the first attempt now in `lb.retry` and `lb.until_timeout`
- Added support for language changes in python 3.8

Removed
- FilenameDict and ConcurrentRunner, which have been deprecated for a while
- `limit_exception_depth`, which is redundant with `hide_in_traceback`
- `lb.Testbed`
- `lb.CommandLineWrapper`, which replaces `lb.ShellBackend`
- `lb.range`, `lb.linspace`, `lb.progress_bar`, which are out of scope and provided by other modules (e.g., tqdm)

Added
Changed
- Explicitly removed support for python 2.x
- Fixes for docstrings in Device.state and Device.settings objects
- All trait type definitions now support the `remap` keyword.
This dictionary value is formed of keys of the validated type of the trait (e.g. lb.Bool should be python bool values)
and values of the desired output that goes "over the wire" to the Device. This generalizes
(and replaces) the functionality that was implemented in the trues_ and falses_ keyword in lb.Bool.
- Added traceback_delay keyword argument to lb.concurrently to configure
whether to immediately display traceback information. This now defaults to
True, which waits to display traceback information until all threads have
finished. When only one thread raises an exception, this reduces superfluous
debug output, which can be overwhelming when lb.concurrently is called in an
inner loop.
- Address a pandas warning about sorting
- Corrected a state and setting bug where setters caused unnecessary extra getter calls
- Fixed the bug in jupyter notebook front panel re-display for objects that already exist
- If Device.state and/or Device.settings are declared with no parent class, automatically subclass from corresponding attribute in the parent class
- Count the number of threads running in lb.concurrently to clear the stop request event, so lb.sleep does not unexpectedly raise ThreadEndedByMaster

Removed

Added
- `labbench.retry` calls a function exception up to a specified number of times
- `labbench.SQLiteLogger.observe_settings` for capturing settings into the database
- `labbench.Email` "device" notifies on disconnection, with info text that includes stderr and any exceptions
- `labbench.sleep` emulates time.sleep, but includes goodies to raise exceptions to end threads at the request of the master thread
- `labbench.until_timeout` (decorator) repeats a function call, suppressing a specified exception until the specified timeout period has expired
- `labbench.kill_by_name` kills a process by name
- `labbench.stopwatch` to time a block of code using a `with` statement
- `labbench.DeviceConnectionLost` exception
- `labbench.check_master` raises ThreadEndedByMaster if the master has requested threads to quit

Changed
- Fixed exception bug in host.py when Host.disconnect() is called after it is already disconnected
- disconnect attribute behavior is now customized for Device subclasses - all disconnect methods in the Device driver MRO are called up to Device.disconnect
- disconnect exceptions are suppressed, though their tracebacks are printed to stderr
- connect attribute behavior is now customized for Device subclasses - all connect methods in the Device driver MRO are starting from Device.connect
- Testbed now includes a stderr attribute, which produces an output log in stderr.log after the Testbed disconnects
- Fix some unicode decode errors that may be raised on special console characters in CommandLineWrapper
- Access to VISADevice.backend now injects the labbench sleep function into the time module, in order to hack thread support into visa queries
- Fixed bugs that sometimes caused duplicated logger output messages on the screen
- VISADevice includes a hack to replace time.sleep with labbench.sleep for responsiveness to exceptions
- labbench.concurrently now supports dictionary inputs, making exceptions more informative
- Device.command_set and Device.command_get have been replaced by the Device.state.getter and Device.state.setter
decorators to follow the same property-like behavior of trait getters and setters
- Automatically generate wrappers for `__init__` methods of device subclasses in order to
help autogenerate documentation and autocomplete in IDEs
- lb.read_relational now skips expanding from files in rows that are '' or None
- lb.panel now supports a testbed keyword to search a Testbed instance for devices instead of the parent namespace
- When only a single thread raises an exception, `labbench.concurrently` now raises that exception instead of `ConcurrentException`
- Fixed a rare race condition in command line execution
- Raise AttributeError on attempts to assign to a state or setting that hasn't been defined
- feather-format is no longer a dependency; it has been replaced with pyarrow
- Use `pyarrow` instead of `feather-format` to implement feather support, reducing the number of dependencies
- Fixed a bug in VISADevice.list_devices

Removed
- CSVLogger may be bitrotten; needs to be checked and possibly deprecated?
- Device.cleanup (it is superceded by the new Device.disconnect behavior)
- Device.setup (superceded by the new Device.connect behavior)
- pythonnet is no longer a required dependency (though it is required if you use lb.DotNetDevice)

Added
- Device.settings for settings traits that are cached locally and do not require communication with the device
- Device.settings.define makes a new settings subclass with adjusted default values

Changed
- Streamlined display of exception tracebacks in concurrent excecution by removing lines with distracing labbench.util internals
- Exceptions raised during connect or disconnect of a Device instance using a `with` block now display information about which instance failed. This should help to reduce confusion debugging failures when connecting multiple devices.
- Simplified the Device.state.connected logic by replacing __getattribute__ munging with a dynamic check for whether Device.backend is a DisconnectedBackend instance or not.
- Bugfixes for CommandLineWrapper
- always and never arguments observe_states are now only applied if they are not empty
- concurrently() now responds to exceptions in the main thread within 100ms, instead of waiting until after the next thread finishes

Removed
- Local state trait classes (local states are now implemented by use of the settings traits)

Added
- LocalList state trait
- support for customizing background processes in CommandLineWrapper via respawn, exception_on_stderr, and no_state_arguments context managers
- refactored all output data munging into a single class, DirectoryMunger, to contain the ugliness that is munging
- new FilenameDict for managing short lists of parameters as single config filenames

Changed
- Local state traits now support the command keyword
- Local state traits now by default support None values
- database .append() now by default makes a deep copy of the passed data,
which is important for thread safety and for cases where a dictionary is reused.
this can be disabled (for example, to improve performance) by passing copy=False
to the append method.
- support waiting for a specified number of queue entries in CommandLineWrapper.read_stdout
- switch to copy instead of move when importing a file into the tree
- debug messages on successful completion of Device setup method after connection
- better detail for debug information in certain rare exceptions in SQLiteLogger
- move imports to __init__ and the root of backends.py to improve threadsafety
- CommandLineWrapper uses subprocess.run instead of subprocess.check_output now for win32 threadsafety
- fixed tests/test_db.py to match current labbench
- moved expensive imports from the top of `backends.py` to `Device.__imports__` method to speed up `labbench` import

Removed