Flexmeasures

.. warning:: Upgrading to this version requires running ``flexmeasures db upgrade`` (you can create a backup first with ``flexmeasures db-ops dump``).

New features
-----------
* Three new CLI commands for cleaning up your database: delete 1) unchanged beliefs, 2) NaN values or 3) a sensor and all of its time series data [see `PR 328 <http://www.github.com/FlexMeasures/flexmeasures/pull/328>`_]
* Add CLI option to pass a data unit when reading in time series data from CSV, so data can automatically be converted to the sensor unit [see `PR 341 <http://www.github.com/FlexMeasures/flexmeasures/pull/341>`_]
* Add CLI option to specify custom strings that should be interpreted as NaN values when reading in time series data from CSV [see `PR 357 <http://www.github.com/FlexMeasures/flexmeasures/pull/357>`_]
* Add CLI commands ``flexmeasures add sensor``, ``flexmeasures add asset-type``, ``flexmeasures add beliefs`` (which were experimental features before) [see `PR 337 <http://www.github.com/FlexMeasures/flexmeasures/pull/337>`_]
* Add CLI commands for showing organisational structure [see `PR 339 <http://www.github.com/FlexMeasures/flexmeasures/pull/339>`_]
* Add a CLI command for showing time series data [see `PR 379 <http://www.github.com/FlexMeasures/flexmeasures/pull/379>`_]
* Add CLI command for attaching annotations to assets: ``flexmeasures add holidays`` adds public holidays [see `PR 343 <http://www.github.com/FlexMeasures/flexmeasures/pull/343>`_]
* Add CLI command for resampling existing sensor data to new resolution [see `PR 360 <http://www.github.com/FlexMeasures/flexmeasures/pull/360>`_]
* Add CLI command to delete an asset, with its sensors and data. [see `PR 395 <http://www.github.com/FlexMeasures/flexmeasures/pull/395>`_]
* Add CLI command to edit/add an attribute on an asset or sensor. [see `PR 380 <http://www.github.com/FlexMeasures/flexmeasures/pull/380>`_]
* Add CLI command to add a toy account for tutorials and trying things [see `PR 368 <http://www.github.com/FlexMeasures/flexmeasures/pull/368>`_]
* Add CLI command to create a charging schedule [see `PR 372 <http://www.github.com/FlexMeasures/flexmeasures/pull/372>`_]
* Support for percent (%) and permille (‰) sensor units [see `PR 359 <http://www.github.com/FlexMeasures/flexmeasures/pull/359>`_]

Bugfixes
-----------

Infrastructure / Support
----------------------
* Plugins can import common FlexMeasures classes (like ``Asset`` and ``Sensor``) from a central place, using ``from flexmeasures import Asset, Sensor`` [see `PR 354 <http://www.github.com/FlexMeasures/flexmeasures/pull/354>`_]
* Adapt CLI command for entering some initial structure (``flexmeasures add structure``) to new datamodel [see `PR 349 <http://www.github.com/FlexMeasures/flexmeasures/pull/349>`_]
* Align documentation requirements with pip-tools [see `PR 384 <http://www.github.com/FlexMeasures/flexmeasures/pull/384>`_]
* Beginning API v3.0 - more REST-like, supporting assets, users and sensor data [see `PR 390 <http://www.github.com/FlexMeasures/flexmeasures/pull/390>`_ and `PR #392 <http://www.github.com/FlexMeasures/flexmeasures/pull/392>`_]

.. warning:: Upgrading to this version requires running ``flexmeasures db upgrade`` (you can create a backup first with ``flexmeasures db-ops dump``).
.. warning:: In case you use FlexMeasures for simulations using ``FLEXMEASURES_MODE = "play"``, allowing to overwrite data is now set separately using :ref:`overwrite-config`. Add ``FLEXMEASURES_ALLOW_DATA_OVERWRITE = True`` to your config settings to keep the old behaviour.
.. note:: v0.8.0 is doing much of the work we need to do to move to the new data model (see :ref:`note_on_datamodel_transition`). We hope to keep the migration steps for users very limited. One thing you'll notice is that we are copying over existing data to the new model (which will be kept in sync) with the `db upgrade` command (see warning above), which can take a few minutes.

New features
-----------
* Bar charts of sensor data for individual sensors, that can be navigated using a calendar [see `PR 99 <http://www.github.com/FlexMeasures/flexmeasures/pull/99>`_ and `PR #290 <http://www.github.com/FlexMeasures/flexmeasures/pull/290>`_]
* Charts with sensor data can be requested in one of the supported [`vega-lite themes <https://github.com/vega/vega-themes#included-themes>`_] (incl. a dark theme) [see `PR 221 <http://www.github.com/FlexMeasures/flexmeasures/pull/221>`_]
* Mobile friendly (responsive) charts of sensor data, and such charts can be requested with a custom width and height [see `PR 313 <http://www.github.com/FlexMeasures/flexmeasures/pull/313>`_]
* Schedulers take into account round-trip efficiency if set [see `PR 291 <http://www.github.com/FlexMeasures/flexmeasures/pull/291>`_]
* Schedulers take into account min/max state of charge if set [see `PR 325 <http://www.github.com/FlexMeasures/flexmeasures/pull/325>`_]
* Fallback policies for charging schedules of batteries and Charge Points, in cases where the solver is presented with an infeasible problem [see `PR 267 <http://www.github.com/FlexMeasures/flexmeasures/pull/267>`_ and `PR #270 <http://www.github.com/FlexMeasures/flexmeasures/pull/270>`_]

.. note:: Read more on these features on `the FlexMeasures blog <https://flexmeasures.io/080-better-scheduling-safer-data/>`__.

Deprecations
------------
* The Portfolio and Analytics views are deprecated [see `PR 321 <http://www.github.com/FlexMeasures/flexmeasures/pull/321>`_]

Bugfixes
-----------
* Fix recording time of schedules triggered by UDI events [see `PR 300 <http://www.github.com/FlexMeasures/flexmeasures/pull/300>`_]
* Set bar width of bar charts based on sensor resolution [see `PR 310 <http://www.github.com/FlexMeasures/flexmeasures/pull/310>`_]
* Fix bug in sensor data charts where data from multiple sources would be stacked, which incorrectly suggested that the data should be summed, whereas the data represents alternative beliefs [see `PR 228 <http://www.github.com/FlexMeasures/flexmeasures/pull/228>`_]

Infrastructure / Support
----------------------
* Account-based authorization, incl. new decorators for endpoints [see `PR 210 <http://www.github.com/FlexMeasures/flexmeasures/pull/210>`_]
* Central authorization policy which lets database models codify who can do what (permission-based) and relieve API endpoints from this [see `PR 234 <http://www.github.com/FlexMeasures/flexmeasures/pull/234>`_]
* Improve data specification for forecasting models using timely-beliefs data [see `PR 154 <http://www.github.com/FlexMeasures/flexmeasures/pull/154>`_]
* Properly attribute Mapbox and OpenStreetMap [see `PR 292 <http://www.github.com/FlexMeasures/flexmeasures/pull/292>`_]
* Allow plugins to register their custom config settings, so that FlexMeasures can check whether they are set up correctly [see `PR 230 <http://www.github.com/FlexMeasures/flexmeasures/pull/230>`_ and `PR #237 <http://www.github.com/FlexMeasures/flexmeasures/pull/237>`_]
* Add sensor method to obtain just its latest state (excl. forecasts) [see `PR 235 <http://www.github.com/FlexMeasures/flexmeasures/pull/235>`_]
* Migrate attributes of assets, markets and weather sensors to our new sensor model [see `PR 254 <http://www.github.com/FlexMeasures/flexmeasures/pull/254>`_ and `project 9 <http://www.github.com/FlexMeasures/flexmeasures/projects/9>`_]
* Migrate all time series data to our new sensor data model based on the `timely beliefs <https://github.com/SeitaBV/timely-beliefs>`_ lib [see `PR #286 <http://www.github.com/FlexMeasures/flexmeasures/pull/286>`_ and `project 9 <http://www.github.com/FlexMeasures/flexmeasures/projects/9>`_]
* Support the new asset model (which describes the organisational structure, rather than sensors and data) in UI and API. Until the transition to our new data model is completed, the new API for assets is at `/api/dev/generic_assets`. [see `PR 251 <http://www.github.com/FlexMeasures/flexmeasures/pull/251>`_ and `PR #290 <http://www.github.com/FlexMeasures/flexmeasures/pulls/290>`_]
* Internal search methods return most recent beliefs by default, also for charts, which can make them load a lot faster [see `PR 307 <http://www.github.com/FlexMeasures/flexmeasures/pull/307>`_ and `PR #312 <http://www.github.com/FlexMeasures/flexmeasures/pull/312>`_]
* Support unit conversion for posting sensor data [see `PR 283 <http://www.github.com/FlexMeasures/flexmeasures/pull/283>`_ and `PR #293 <http://www.github.com/FlexMeasures/flexmeasures/pull/293>`_]
* Improve the core device scheduler to support dealing with asymmetric efficiency losses of individual devices, and with asymmetric up and down prices for deviating from previous commitments (such as a different feed-in tariff) [see `PR 291 <http://www.github.com/FlexMeasures/flexmeasures/pull/291>`_]
* Stop automatically triggering forecasting jobs when API calls save nothing new to the database, thereby saving redundant computation [see `PR 303 <http://www.github.com/FlexMeasures/flexmeasures/pull/303>`_]

This is a patch release in the 0.7.x series and includes some regression fixes. We recommend that all users upgrade to this version.

See our [changelog](https://flexmeasures.readthedocs.io/en/stable/changelog.html#v0-7-1-november-8-2021) for a list of all the changes.

.. warning:: The config setting ``FLEXMEASURES_PLUGIN_PATHS`` has been renamed to ``FLEXMEASURES_PLUGINS``. The old name still works but is deprecated.

New features
-----------
* Set a logo for the top left corner with the new FLEXMEASURES_MENU_LOGO_PATH setting [see `PR 184 <http://www.github.com/SeitaBV/flexmeasures/pull/184>`_]
* Add an extra style-sheet which applies to all pages with the new FLEXMEASURES_EXTRA_CSS_PATH setting [see `PR 185 <http://www.github.com/SeitaBV/flexmeasures/pull/185>`_]
* Data sources can be further distinguished by what model (and version) they ran [see `PR 215 <http://www.github.com/SeitaBV/flexmeasures/pull/215>`_]
* Enable plugins to automate tests with app context [see `PR 220 <http://www.github.com/SeitaBV/flexmeasures/pull/220>`_]

.. note:: Read more on these features on `the FlexMeasures blog <https://flexmeasures.io/070-professional-plugins/>`__.

Bugfixes
-----------
* Fix users resetting their own password [see `PR 195 <http://www.github.com/SeitaBV/flexmeasures/pull/195>`_]
* Fix scheduling for heterogeneous settings, for instance, involving sensors with different time zones and/or resolutions [see `PR 207 <http://www.github.com/SeitaBV/flexmeasures/pull/207>`_]
* Fix ``sensors/<id>/chart`` view [see `PR 223 <http://www.github.com/SeitaBV/flexmeasures/pull/223>`_]

Infrastructure / Support
----------------------
* FlexMeasures plugins can be Python packages now. We provide `a cookie-cutter template <https://github.com/SeitaBV/flexmeasures-plugin-template>`_ for this approach. [see `PR #182 <http://www.github.com/SeitaBV/flexmeasures/pull/182>`_]
* Set default timezone for new users using the FLEXMEASURES_TIMEZONE config setting [see `PR 190 <http://www.github.com/SeitaBV/flexmeasures/pull/190>`_]
* To avoid databases from filling up with irrelevant information, only beliefs data representing *changed beliefs are saved*, and *unchanged beliefs are dropped* [see `PR 194 <http://www.github.com/SeitaBV/flexmeasures/pull/194>`_]
* Monitored CLI tasks can get better names for identification [see `PR 193 <http://www.github.com/SeitaBV/flexmeasures/pull/193>`_]
* Less custom logfile location, document logging for devs [see `PR 196 <http://www.github.com/SeitaBV/flexmeasures/pull/196>`_]
* Keep forecasting and scheduling jobs in the queues for only up to one day [see `PR 198 <http://www.github.com/SeitaBV/flexmeasures/pull/198>`_]

This is a patch release in the 0.6.x series and includes some regression fixes. We recommend that all users upgrade to this version.

See our [changelog](https://flexmeasures.readthedocs.io/en/stable/changelog.html#v0-6-1-october-23-2021) for a list of all the changes.

The two most notable new features are that users and assets now belong to accounts (allowing for multi-tenancy setups), and that it's now possible to monitor for things going wrong (and be able to react in time).


.. warning:: Upgrading to this version requires running ``flexmeasures db upgrade`` (you can create a backup first with ``flexmeasures db-ops dump``).
In case you are using experimental developer features and have previously set up sensors, be sure to check out the upgrade instructions in `PR 157 <https://github.com/SeitaBV/flexmeasures/pull/157>`_. Furthermore, if you want to create custom user/account relationships while upgrading (otherwise the upgrade script creates accounts based on email domains), check out the upgrade instructions in `PR #159 <https://github.com/SeitaBV/flexmeasures/pull/159>`_. If you want to use both of these custom upgrade features, do the upgrade in two steps. First, as described in PR 157 and upgrading up to revision b6d49ed7cceb, then as described in PR 159 for the rest.

.. warning:: The config setting ``FLEXMEASURES_LISTED_VIEWS`` has been renamed to ``FLEXMEASURES_MENU_LISTED_VIEWS``.

.. warning:: Plugins now need to set their version on their module rather than on their blueprint. See the `documentation for writing plugins <https://flexmeasures.readthedocs.io/en/v0.6.0/dev/plugins.html>`_.

New features
-----------
* Multi-tenancy: Supporting multiple customers per FlexMeasures server, by introducing the `Account` concept. Accounts have users and assets associated. [see `PR 159 <http://www.github.com/SeitaBV/flexmeasures/pull/159>`_ and `PR #163 <http://www.github.com/SeitaBV/flexmeasures/pull/163>`_]
* In the UI, the root view ("/"), the platform name and the visible menu items can now be more tightly controlled (per account roles of the current user) [see also `PR 163 <http://www.github.com/SeitaBV/flexmeasures/pull/163>`_]
* Analytics view offers grouping of all assets by location [see `PR 148 <http://www.github.com/SeitaBV/flexmeasures/pull/148>`_]
* Add (experimental) endpoint to post sensor data for any sensor. Also supports our ongoing integration with data internally represented using the `timely beliefs <https://github.com/SeitaBV/timely-beliefs>`_ lib [see `PR #147 <http://www.github.com/SeitaBV/flexmeasures/pull/147>`_]

.. note:: Read more on these features on `the FlexMeasures blog <https://flexmeasures.io/v060-multi-tenancy-error-monitoring/>`__.

Bugfixes
-----------

Infrastructure / Support
----------------------
* Add possibility to send errors to Sentry [see `PR 143 <http://www.github.com/SeitaBV/flexmeasures/pull/143>`_]
* Add CLI task to monitor if tasks ran successfully and recently enough [see `PR 146 <http://www.github.com/SeitaBV/flexmeasures/pull/146>`_]
* Document how to use a custom favicon in plugins [see `PR 152 <http://www.github.com/SeitaBV/flexmeasures/pull/152>`_]
* Allow plugins to register multiple Flask blueprints [see `PR 171 <http://www.github.com/SeitaBV/flexmeasures/pull/171>`_]
* Continue experimental integration with `timely beliefs <https://github.com/SeitaBV/timely-beliefs>`_ lib: link multiple sensors to a single asset [see `PR #157 <https://github.com/SeitaBV/flexmeasures/pull/157>`_]
* The experimental parts of the data model can now be visualised, as well, via `make show-data-model --uml --dev` [also in `PR 157 <https://github.com/SeitaBV/flexmeasures/pull/157>`_]