Muninn

~~~~~~~~~~~~~~

* Changed license of muninn to BSD.

* Muninn can now be used as a pure catalogue server.
An archive root directory is still needed, but can remain empty.

* Changed the core schema:
- core.archive_date is now optional and is only set if core.archive_path is
set.
It will reflect the time at which the product was stored in the archive.
- added core.metadata_date which represents the last modification time of
the metadata record. It will automatically be updated if the metadata is
changed via the muninn interfaces.
- added optional core.remote_url field which can contain urls of the form
file://, http://, ftp://, etc. that point to a remote location of the
product.

* The schema_name option for the postgresql backend has been removed.
This is now replaced by a table_prefix option that is applicable for
both the postgresql and sqlite backends.

* The sqlite backend will no longer remove the database file for a 'destroy'
(it will only remove the tables). Both 'prepare' and 'destroy' can be
performed on an existing sqlite database file.

* Added 'pull' command that allows downloading of products that are not yet
in the archive. It will use the core.remote_url value to perform the
download. A 'retrieve' still only works on products that are already in the
archive.
The pull method supports the default http/https/file/ftp protocols but also
supports a custom ecmwfapi protocol that will use the ecmwf-api-client
python package to download data from the ECMWF MARS archive.

* The muninn Struct type that is used in the Python interface to store
metadata can now be initialized with a Python dict.

* Added create_properties() and generate_uuid() functions to the Python
muninn.Archive interface. These are functions that should be used to create
pure metadata records (without archived product) in a muninn archive.

* Added verify_hash() function to the Python muninn.Archive interface.

* muninn.Archive.generate_uuid() is now a static method.

* Improved preparing and destroying of muninn archives. This will now better
distinguish between the archive directory and the catalogue.

* Verifying of the hash after an ingest of a product in the archive is now
optional (--verify-hash argument to muninn-ingest).

* The muninn sqlite backend now adds a spatialindex on footprint.
The indices for both backends have all been renamed to idx_<table>_<field>.

* muninn-prepare has a --dry-run mode, that just dumps the SQL that would be
executed.

* The muninn sqlite backend now supports the escape character '\' when using
the ~= operator.

* Moved change history from README to separate CHANGELOG file.
Renamed README.developer to EXTENSIONS and added DEVELOPER file.
Include all documentation files with the source package.

~~~~~~~~~~~~~~~~

* Fixed parsing of negative coordinates in literal geometry values.

~~~~~~~~~~~~~~

* Added sqlite backend.

* Added arithmetic operators "+", "-", "*", and "/" for all numeric types.

* Added binary "-" operator for timestamps. This operator returns the length
of the time interval between two timestamps as a fractional number of
seconds.

* Updated comparison operators to support comparisons between integer and real
types.

* Changed the muninn expression language to use a sub-set of the Well Known
Text (WKT) markup language to represent literal geometry values.

* Added -c/--catalogue-only arguments to prepare/destroy commands to only
create/remove the catalogue database (while keeping the product archive
intact).

* If a product that is ingested is already in the right target location in
the archive on disk, muninn will just leave the product at its location
without trying to move or symlink it.

* Added 'id' primary key to links and tags tables, which is to support other
front-ends (which don't support multi-field primary keys) on top of the
same muninn database.
This change should not require a migration of existing databases (as long
as no other front-ends are planned to be used).

* Tags no longer have the constraint that they should be an 'identifier'.
Added ',' as tag separator to output of muninn-list-tags.

* Changed the Postgresql backend to create indices on most columns of the
'core', 'tag', and 'link' tables when creating these tables.

* MUNINN_CONFIG_PATH can now also refer directly to archive configuration
files.

~~~~~~~~~~~~~~~~

* Changed the tag() function of the Archive class such that setting a tag on
a product that already has that tag is not treated as an error.

* Changed the link() function of the Archive class such that adding a link
to a product that already has that link is not treated as an error.

* Changed the tags() function of the Archive class to return a list of tags
ordered by tag name.

* Improved error reporting for errors related to the creation or destruction
of an archive.

* Improved error reporting by the Postgresql backend.

~~~~~~~~~~~~~~

* Renamed property core.logical_name to core.product_name.

* Which namespaces and product types are available is now configured per
archive (in the archive configuration file) instead of globally.

* Extensions have been split into two types: namespace extensions (that
contain namespace definitions) and product type extensions (that contain
product type plug-ins). Extensions no longer have to contain both.

* Extensions should now be reachable via the PYTHONPATH, the
MUNINN_EXTENSION_PATH is no longer in use.

* Added core.size property to contain the product size on disk (in bytes).
When a product is ingested, the size of the product will be computed and
stored automatically.

* Added a count option to muninn-search that prints the number of products
found instead of the normal output.

* Added a summary options to muninn-search that prints a short summary of the
products found instead of the normal output.

* Changed the retrieve(), export(), remove() and strip() functions of the
Archive class to return the number of affected products.

* Added convenience functions for the retrieve(), export(), remove() and
strip() functions of the Archive class that perform the desired function
given a uuid or product name instead of a muninn expression.

* Added the product_path() function to the Archive class that returns the path
to a product given a uuid or product name.

* Added the root() function to the Archive class that returns the archive root
path.

* Removed property decorators from functions of the Archive class.

* Removed dependency on importlib and added compatibility for older versions
of psycopg (>=2.0) and Python (>=2.6).

* Added documentation for muninn extension developers (see README.developer).

~~~~~~~~~~~~~~~~

* Changed functions that operate on product data, i.e. export(), retrieve(),
remove(), strip(), and rebuild_properties() to raise an exception when
called on a partially ingested product (of which the core.active property
equals false).

* Added a "force" keyword argument to the remove() function that enables the
removal of partially ingested products.

* Added a -f/--force option to the muninn-remove tool that enables the removal
of partially ingested products.

* Applied minor correction to update instructions for version 1.3.