Pyde1

Latest version: v2.0.0

Safety actively analyzes 710445 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 5 of 6

0.6.0

------------------

**The Mimoja Release**

I am not sure how / where to store shots and profiles. I hate it to
only have it browser local.

*So do I. Wonder no longer.*

New
===

A SQLite3 database now saves all profiles uploaded to the DE1, as well
as capturing virtually all real-time data during all flow sequences,
including a brief set of data from *before* the state transition.

Profiles are unique by the content of their "raw source" and also have a
"fingerprint" that is common across all profiles that produce the same
"program" for the DE1. Changing a profile's name alone does not change
this fingerprint. Changing the frames in a profile without changing the
name changes both the ID of the profile, as well as its fingerprint.
These are both calculated using SHA1 from the underlying data, so should
be consistent across installs for the same source data or frame set.

Profiles can also be searched by the customary metadata:

- Title
- Author
- Notes
- Beverage type
- Date added

``aiosqlite`` and its dependencies are now required.

Legacy-style shot data can be extracted from the database by an
application other than that which is running the DE1. Creating a
Visualizer-compatible "file" for upload can be done in around 80-100 ms
on a RPi 3B. If written to a physical file, it is also compatible with
John Weiss' shot-plotting programs. See ``pyDE1/shot_file/legacy.py``

The database retains the last-known profile uploaded to the DE1. If a
flow sequence beings prior to uploading a profile, it is used as the
"most likely" profile and identified in the database with the
``profile_assumed`` flag.

.. note::
The database needs to be manually initialized prior to use.

One approach is

::

sudo -u <user> sqlite3 /var/lib/pyDE1/pyDE1.sqlite3 \
< path/to/pyDE1/src/pyDE1/database/schema/schema.001.sql

Changed
=======

Upload limit changed to 16 kB to accommodate larger profiles.

FlowSequencer events are now notified over ``SequencerGateNotification``
and include a ``sequence_id`` and the ``active_state`` for use with
history logging.

``Profile.from_json()`` now expects a string or bytes-like object,
rather than a dict. This change is to ease capture of the profile
"source" for use with history logging.

``ProfileByFrames.from_json()`` no longer rounds the floats to maintain
the integrity of the original source. They will still be rounded at the
time that they are encoded into binary payloads.

Standard initialization of the DE1 now includes reading
``CUUID.Versions`` and ``ShotSettings`` to speed first-time store of
profiles.

Robustness of shutdown improved.

Internal ``Profile`` class extended to capture "raw source", metadata,
and UUIDs for both the raw source and the resulting "program" sent to
the DE1.

Fixed
=====

In ``find_first_and_load.py``, ``set_saw()`` now uses the passed mass

Deprecated
==========

``Profile.from_json_file()`` as it is no longer needed with the API able
to upload profiles. If needed within the code base, read the file, and
pass to ``Profile.from_json()`` to ensure that the profile source and
signatures are properly updated.

``DE1._recorder_active`` and the contents of ``shot_file.py`` have been
superseded by database logging.

Known Issues
============

The database name is hard-coded at this time.

``Profile.regenerate_source()`` is not implemented at this time.

Occasionally, during shutdown, the database capture reports that it was
passed ``None`` and an exception is raised. This may be due to shut
down, or may be due to failure to retrieve an earlier exception from the
task.


------------------

0.5.0

------------------

New
===

Bluetooth scanning with API. See ``README.bluetooth.md`` for details

API can set scale and DE1 by ID, by first\_if\_found, or None

A list of logs and individual logs can be obtained with GET
``Resource.LOGS`` and ``Routine.LOG``

``ConnectivityEnum.READY`` added, allowing clients to clearly know if
the DE1 or scale is available for use.

.. warning::
Previous code that assumed that ``.CONNECTED`` was the
terminal state should be modified to recognize ``.READY``.

``examples/find_first_and_load.py`` demonstrates stand-alone connection
to a DE1 and scale, loading of a profile, setting of shot parameters,
and disconnecting from these devices.

``scale_factory(BLEDevice)`` returns an appropriate ``Scale`` subtype

``Scale`` subtypes need to register their advertisement-name prefix,
such as

::

Scale.register_constructor(AtomaxSkaleII, 'Skale')

Timeout on ``await`` calls initiated by the API

Use of connecting to the first-found DE1 and scale, monitoring MQTT,
uploading a profile, setting SAW, all through the API is shown in
``examples/find_first_and_load.py``

Example profiles: EB6 has 30-s ramp vs EB5 at 25-s

Add ``timestamp_to_str_with_ms()`` to ``pyDE1.utils``

On an error return to the inbound API, an exception trace is provided,
when available. This is intended to assist in error reporting.


Changed
=======

HTTP API PUT/PATCH requests now return a list, which may be empty.
Results, if any, from individual setters are returned as dict/obj
members of the list.

Some config parameters moved into ``pyDE1.config.bluetooth``

"find\_first" functionality now implemented in ``pyDE1.scanner``

``de1.address()`` is replaced with ``await de1.set_address()`` as it
needs to disconnect the existing client on address change. It also
supports address change.

``Resource.SCALE_ID`` now returns null values when there is no scale.

There's not much left of ``ugly_bits.py`` as its functions now should be
able to be handled through the API.

On connect, if any of the standard register reads fails, it is logged
with its name, and retried (without waiting).

An additional example profile was added. EB6 has 30-s ramp vs EB5 at
25-s. Annoying rounding errors from Insight removed.

0.4.1

------------------

Fixed
=====

Import problems with ``manual_setup`` resolved with an explicit
reference to the ``pyDE1.ugly_bits`` version. Local overrides that may
have been in use prior will likely no longer used. TODO: Provide a more
robust config system to replace this.

Non-espresso flow (hot water flush, steam, hot water) now have their
accumulated volume associated with Frame 0, rather than the last frame
number of the previous espresso shot.


------------------

0.4.0

------------------

New
===

Support for non-GHC machines to be able to start flow through the API

More graceful shutdown on SIGINT, SIGQUIT, SIGABRT, and SIGTERM

Logging to a single file, ``/tmp/log/pyDE1/combined.log`` by default. If
changed to, for example, ``/var/log/pyDE1/``, the process needs write
permission for the directory.

.. note::
Keeping the logs in a dedicated directory is suggested, as the
plan is to provide an API where a directory list will be used to
generate the ``logs`` collection. ``/tmp/`` is used for ease of
development and is not guaranteed to survive a reboot.

Log file is closed and reopened on SIGHUP.

Long-running processes, tasks, and futures are supervised, with
automatic restart should they unexpectedly terminate. A limit of two
restarts is in place to prevent "thrashing" on non-transient errors.

Changed
=======

Exceptions moved into ``pyDE1.exceptions`` for cleaner imports into
child processes.

String-generation utilities moved from ``pyDE1.default_logger`` into
``pyDE1.utils``

- ``data_as_hex()``
- ``data_as_readable()``
- ``data_as_readable_or_hex()``

Remove inclusion of ``pyDE1.default_logger`` and replace with explicit
calls to ``initialize_default_logger()`` and
``set_some_logging_levels()``

Change from ``asyncio-mqtt`` to "bare" ``paho-mqtt``. The
``asyncio-mqtt`` module is still a requirement as it is used in
``examples/monitor_delay.py``

Controller now runs in its own process. Much of what was in
``try_de1.py`` is now in ``controller.py``

Log entries now include the process name.

IPC between the controller and outbound (MQTT) API now uses a pipe and
``loop.add_reader()`` to improve robustness and ease graceful shutdown.

Several internal method signatures changed to accommodate changes in
IPC. These are considered "internal" and do not impact the two, public
APIs.

Significant refactoring to move setup and run code out of ``try_de1.py``
and into more appropriate locations. The remaining "manual" setup steps
are now in ``ugly_bits.py``. See also ``run.py``

0.3.0

------------------

New
===

Upload of profile (JSON "v2" format) available with PUT at de1/profile

curl -D - -X PUT --data examples/jmk\_eb5.json
http://localhost:1234/de1/profile

Line frequency GET/PATCH at de1/calibration/line\_frequency implemented.
Valid values are 50 or 60. This does not impact the DE1, only if 1/100
or 1/120 is used to calculate volume dispensed.

The HTTP API now checks to see if the request can be serviced with the
current DE1 and Scale connectivity. This should help enable people that
don't have a Skale II connected.

.. note:
Although the DE1 and Scale can be reconnected, they are not
reinitialized at this time.

``BleakClientWrapped.willful_disconnect`` property can be used to
determine if the on-disconnect callback was called as a result of an
intentional (locally initiated) or unintentional disconnect.

``BleakClientWrapped.name`` provides the advertised device name under
BlueZ and should not fail under macOS (or Windows).

Changed
=======

0.2.0

------------------

Inbound Control and Query API
=============================

An inbound API has been provided using a REST-like interface over HTTP.
The API should be reasonably complete in its payload and method
definitions and comments are welcomed on its sufficiency and
completeness.

Both the inbound and outbound APIs run in separate *processes* to reduce
the load on the controller itself.

GET should be available for the registered resources. See, in
``src/pyDE1/dispatcher``

- ``resource.py`` for the registered resources, and
- ``mapping.py`` for the elements they contain, the expected value
types, and how they nest.

``None`` or ``null`` are often used to me "no value", such as for
stop-at limits. As a result, though similar, this is not an `RFC7368
JSON Merge Patch <https://datatracker.ietf.org/doc/html/rfc7386>`__.

In Python notation, ``Optional[int]`` means an ``int`` or ``None``.
Where ``float`` is specified, a JSON value such as ``20`` is permitted.

GET presently returns "unreadable" values to be able to better show the
structure of the JSON. When a value is unreadable, ``math.nan`` is used
internally, which is output as the JSON ``NaN`` token.

GET also returns empty nodes to illustrate the structure of the
document. This can be controlled with the ``PRUNE_EMPTY_NODES`` variable
in ``implementation.py``

Although PATCH has been implemented for most payloads, PUT is not yet
enabled. PUT will be the appropriate verb for\ ``DE1_PROFILE`` and
``DE1_FIRMWARE`` as, at this time, in-place modification of these is not
supported. The API mechanism for starting a firmware upload as not been
determined, as it should be able to abort as it runs in the background,
as well as notify when complete. Profile upload is likely to be similar,
though it occurs on a much faster timescale.

If you'd like the convenience of a GET of the same resource after a
PATCH, you can set ``READ_BACK_ON_PATCH`` to ``True`` in
``dispacher.py``

The Python ``http.server`` module is used. It is not appropriate for
exposed use. There is no security to the control and query API at
this time. See further
https://docs.python.org/3/library/http.server.html

It is likely that the server, itself, will be moved to a uWSGI (or
similar) process.

With either the present HTTP implementation or a future uWSGI one, use
of a webserver, such as ``nginx``, will be able to provide TLS,
authentication, and authorization, as well as a more "production-ready"
exposure.

Other Significant Changes
=========================

- ``ShotSampleWithVolumeUpdates`` (v1.1.0) adds ``de1_time``.
``de1_time`` and ``scale_time`` are preferred over ``arrival_time``
as, in a future version, these will be estimates that remove some of
the jitter relative to packet-arrival time.

- To be able to keep cached values of DE1 variables current, a
read-back is requested on each write.

- ``NoneSet`` and ``NONE_SET`` added to some ``enum.IntFlag`` to
provide clearer representations

- Although ``is_read_once`` and ``is_stable`` have been roughed in,
optimizations using them have not been done

- Disabled reads of ``CUUID.ReadFromMMR`` as it returns the request
itself, which is not easily distinguishable from the data read. These
two interpret their ``Length`` field differently, making it difficult
to determine if ``5`` is an unexpected value or if it was just that 6
words were requested to be read.

- Scaling on ``MMR0x80LowAddr.TANK_WATER_THRESHOLD`` was corrected.


------------------

Page 5 of 6

© 2025 Safety CLI Cybersecurity Inc. All Rights Reserved.