Trollflow-sat

--------------------
- Update changelog. [Panu Lahtinen]
- Bump version: 0.11.1 → 0.12.0. [Panu Lahtinen]
- Merge pull request 41 from pnuu/bugfix-null-writer. [Panu Lahtinen]

Bugfix default format settings
- Do not force a writer, default to GeoTIFF if pattern doesn't have a
format. [Panu Lahtinen]
- Skip dataset if writing fails. [Panu Lahtinen]
- Merge pull request 35 from vgiuffrida/master. [Panu Lahtinen]

Add writer extra args loading from configuration file
- Added writer configuration function. [root]
- Merge pull request 38 from mraspaud/fix-saving-crash. [Panu Lahtinen]

Fix format options being replaced by defaults when they evaluate to False
- Fix replacing fill_value=0 with None. [Martin Raspaud]
- Add more information when writer crashes. [Martin Raspaud]
- Log exception during saving. [Martin Raspaud]
- Merge pull request 40 from pnuu/bugfix-coverage-instrument. [Panu
Lahtinen]

Make sure to get the full instrument name for Pass()
- Make sure to get the full instrument name for Pass() [Panu Lahtinen]
- Merge pull request 39 from pnuu/patch-add-reduce_data-kwarg. [Panu
Lahtinen]

Add 'reduce_data' kwarg to satpy_resampler
- Add 'reduce_data' kwarg to satpy_resampler. [Panu Lahtinen]

--------------------
- Update changelog. [Panu Lahtinen]
- Bump version: 0.11.0 → 0.11.1. [Panu Lahtinen]
- Merge pull request 34 from pytroll/bugfix-pattern-time-name. [Panu
Lahtinen]

Use new public functions from trollsift to get the pattern parts
- Use new public functions from trollsift to get the pattern parts.
[Panu Lahtinen]
- Merge pull request 33 from pytroll/feature-save-overviews. [Panu
Lahtinen]

Embed overviews to saved images
- Move add_overviews() to trollflow_sat.utils. [Panu Lahtinen]
- Add unittests for overview addition. [Panu Lahtinen]
- Return if import fails. [Panu Lahtinen]
- Fix reading filenames from messages, add docstring to add_overviews()
[Panu Lahtinen]
- Catch exception raised by unsupported files. [Panu Lahtinen]
- Use context manager to handle file opening. [Panu Lahtinen]
- Add a possibility to embed reduced sized images "overviews" [Panu
Lahtinen]

--------------------
- Update changelog. [Panu Lahtinen]
- Bump version: 0.10.0 → 0.11.0. [Panu Lahtinen]
- Merge pull request 30 from pytroll/develop. [Panu Lahtinen]

Add Python 3 compatibility, drop mpop support
- Pass save settings directly as kwargs to save_datasets() [Panu
Lahtinen]
- Change filename_pattern kwarg to filename. [Panu Lahtinen]
- Add log message if instrument alias is used. [Panu Lahtinen]
- Add aliases to translate sensor name in message to SatPy naming. [Panu
Lahtinen]
- Fix test for queue size. [Panu Lahtinen]
- Adjust tests for the fixed area-by-area processing. [Panu Lahtinen]
- Fix bug that resulted in processing each area several times. [Panu
Lahtinen]
- Add config where process_by_area is False. [Panu Lahtinen]
- Move process_by_area from flow config to product config. [Panu
Lahtinen]
- Add six to requirements. [Panu Lahtinen]
- Update install and test requirements. [Panu Lahtinen]
- Add more unit tests for utility functions. [Panu Lahtinen]
- Remove unnecessary line-split. [Panu Lahtinen]
- Remove unnecessary try/except. [Panu Lahtinen]
- Update variable names. [Panu Lahtinen]
- Fix mocked function name. [Panu Lahtinen]
- Rename utility function. [Panu Lahtinen]
- Remove obsolete utility functions. [Panu Lahtinen]
- Use OrderedDicts, add MockDset class, handle needs for writer tests.
[Panu Lahtinen]
- Add unit tests for satpy_writer. [Panu Lahtinen]
- Add satpy writer tests to test suite. [Panu Lahtinen]
- Ensure create_fnames() always returns two items. [Panu Lahtinen]
- Use ordered_load from trollflow to read YAML files. [Panu Lahtinen]
- Make sure there's a thread before doing a join() [Panu Lahtinen]
- Ensure save settings is a dictionary. [Panu Lahtinen]
- Delete written files in tearDown() [Panu Lahtinen]
- Add satpy_resampler tests to test suite. [Panu Lahtinen]
- Add unit tests for satpy_resampler. [Panu Lahtinen]
- Add two more product configs, add platform_name to basic config. [Panu
Lahtinen]
- Add "common" section to product config. [Panu Lahtinen]
- Add resample() method. [Panu Lahtinen]
- Remove more references to non-xarray metadata dictionaries. [Panu
Lahtinen]
- Remove try/excepts of obsolete non-xarray metadata structures. [Panu
Lahtinen]
- Move MockScene to utils.py. [Panu Lahtinen]
- Move common things to utils.py. [Panu Lahtinen]
- Rename patched objects with better names. [Panu Lahtinen]
- Add satpy_compositor tests to test suite. [Panu Lahtinen]
- Fix mock import for Py2, remove calls to `assert_called_once()` [Panu
Lahtinen]
- Add more satpy_compositor unit tests. [Panu Lahtinen]
- Handle readers better, ensure global_data instance is created. [Panu
Lahtinen]
- Add tests for SceneLoader. [Panu Lahtinen]
- Add placeholders for satpy plugins. [Panu Lahtinen]
- Merge pull request 32 from pytroll/bugfix-urlparse. [Panu Lahtinen]

Use urlparse to guarantee local path
- Fix urlparse import. [Panu Lahtinen]
- Use urlparse to guarantee local path. [Panu Lahtinen]
- Fix test due to changed default save format. [Panu Lahtinen]
- Merge pull request 31 from pytroll/feature-fill_value. [Panu
Lahtinen]

Better handling of save format settings
- Use default format setting if none is configured. [Panu Lahtinen]
- Refactor save format setting handling. [Panu Lahtinen]
- Read fill values from the product config. [Panu Lahtinen]
- Update dict used to compose message topic, again. [Panu Lahtinen]
- Merge branch 'develop' of https://github.com/pytroll/trollflow-sat
into develop. [Panu Lahtinen]
- Merge pull request 29 from pytroll/feature-lazy. [Panu Lahtinen]

Collect areas and composites together for processing
- Add more metadata that can be used in topic compose() [Panu Lahtinen]
- Reorder logging to reduce output. [Panu Lahtinen]
- Fix config example: do not use locking in compositor. [Panu Lahtinen]
- Fix lock release/acquisition in resampler. [Panu Lahtinen]
- Fix queue import, restructure and clean code. [Panu Lahtinen]
- Check if trollsched is available. [Panu Lahtinen]
- Add use_locks to instance attributes and set default to False. [Panu
Lahtinen]
- Restructure and clean invoke() [Panu Lahtinen]
- Remove mention of external calibration. [Panu Lahtinen]
- Remove grouping by area. [Panu Lahtinen]
- Remove obsolete .info attribute access. [Panu Lahtinen]
- Update and clarify example configs. [Panu Lahtinen]
- Make it possible to process one area at a time. [Panu Lahtinen]
- Fix log message copy&paste typo. [Panu Lahtinen]
- Make config string unicode. [Panu Lahtinen]
- Update example configs for current SatPy version. [Panu Lahtinen]
- Collect all areas and composites together for optimized processing.
[Panu Lahtinen]
- Use simple_image writer as default. [Panu Lahtinen]
- Merge pull request 28 from pytroll/feature-python3-support. [Panu
Lahtinen]

Python 3 compatibility
- Add Python 3 to Travis tests. [Panu Lahtinen]
- Use six to import queue. [Panu Lahtinen]
- Fix StringIO import, reflow long lines. [Panu Lahtinen]
- Cast dict.items() to tuple for Python 3 support. [Panu Lahtinen]
- Remove mpop and mipp from requirements. [Panu Lahtinen]
- Remove mpop plugins and example config files. [Panu Lahtinen]
- Remove mpop import. [Panu Lahtinen]
- Add _template to the filename. [Panu Lahtinen]
- Update dict used to compose message topic. [Panu Lahtinen]
- Merge pull request 26 from pytroll/feature-reader-config. [Panu
Lahtinen]

Optionally give readers in the flow configuration file
- Read readers to try from flow configuration file, update example
config. [Panu Lahtinen]
- Merge pull request 24 from pytroll/feature-separate-extra-metadata.
[Panu Lahtinen]

Separate scene and additional information
- Use productname returned by utils.create_fnames() [Panu Lahtinen]
- Use the correct metadata dictionary for area_id. [Panu Lahtinen]
- Add area ID to scene attrs/info instead of external metadata. [Panu
Lahtinen]
- Merge branch 'feature-separate-extra-metadata' of
https://github.com/pytroll/trollflow-sat into feature-separate-extra-
metadata. [Panu Lahtinen]
- Separate scene and additional information. [Panu Lahtinen]
- Separate scene and additional information. [Panu Lahtinen]
- Merge pull request 23 from pytroll/fix-fetch-collections. [Panu
Lahtinen]

Add support for fetching collections
- Bugfix filenames missing for collection. [Martin Raspaud]
- Support collections in compositor. [Martin Raspaud]
- Allow collections of simple files. [Martin Raspaud]
- Add support for fetching collections. [Martin Raspaud]
- Merge pull request 22 from ch-k/feature-satpywriter-complex-
publishvars. [Panu Lahtinen]

Complex expressions for SatpyWrite publish_vars
- Support for dpath expressions in publish_vars. [Christian Kliche]

example:

publish_vars:
source_uri: "/dataset/*/uri"

This creates a list of the original URIs and publishs it as source_uri.

- Complex expressions for SatpyWrite publish_vars. [Christian Kliche]

It is now possible to use python expressions to forward more complex values:

publish_vars:
source_uri: "=[e.get('uri') for e in info['dataset']]"

This creates a list of the original URIs and publishs it as source_uri.

Conflicts:
examples/flow_processor_satpy.yaml_template
trollflow_sat/satpy_writer.py

- Comment out adaguc writer, add coverage_check config option. [Panu
Lahtinen]
- Update changelog. [Panu Lahtinen]
- Bump version: 0.9.0 → 0.10.0. [Panu Lahtinen]
- Merge pull request 21 from pytroll/feature-ignore-message-items.
[Panu Lahtinen]

Add ignore_* functionnality for messages items
- Comment out ignore_ parameter in example workflow. [Martin Raspaud]
- Add ignore_* functionnality for messages items. [Martin Raspaud]
- Merge pull request 17 from pytroll/bugfix-xarray. [Panu Lahtinen]

Fix compatibility with satpy/feature-xarray
- Merge branch 'develop' into bugfix-xarray. [Martin Raspaud]
- Add an option to completely skip coverage checking. [Panu Lahtinen]
- Merge pull request 16 from pytroll/bugfix-xarray. [Martin Raspaud]

Fix compatibility with satpy/feature-xarray
- Merge pull request 15 from pytroll/develop. [Panu Lahtinen]

Merge develop to master
- Fix satpy_resampler to support xarray. [Martin Raspaud]
- Fix compatibility with satpy/feature-xarray. [Martin Raspaud]
- Fix compatibility with satpy/feature-xarray. [Martin Raspaud]

-------------------
- Update changelog. [Panu Lahtinen]
- Bump version: 0.8.0 → 0.9.0. [Panu Lahtinen]
- Merge pull request 14 from pytroll/fix-saving-removed. [Panu
Lahtinen]

Avoid crashing when a composite has been removed
- Check that the configure composite is still available in the Scene.
[Panu Lahtinen]
- Merge pull request 9 from pytroll/fix-delayed. [Panu Lahtinen]

Fix processing of delayed datasets
- Merge branch 'fix-delayed' of https://github.com/pytroll/trollflow-sat
into fix-delayed. [Panu Lahtinen]
- Merge branch 'fix-delayed' of https://github.com/pytroll/trollflow-sat
into fix-delayed. [Panu Lahtinen]
- Add mask_area kwarg, add comments. [Panu Lahtinen]
- Expose "mask_area" kwarg. [Panu Lahtinen]
- Remove dataset IDs from the scene info as unecessary. [Panu Lahtinen]
- Fix handling of "delayed" datasets. [Panu Lahtinen]
- Fix topic of message for new files. [Panu Lahtinen]
- Add mask_area kwarg, add comments. [Panu Lahtinen]
- Expose "mask_area" kwarg. [Panu Lahtinen]
- Remove dataset IDs from the scene info as unecessary. [Panu Lahtinen]
- Fix handling of "delayed" datasets. [Panu Lahtinen]
- Fix topic of message for new files. [Panu Lahtinen]
- Remove dataset IDs from the scene info as unecessary. [Panu Lahtinen]
- Add mask_area kwarg, add comments. [Panu Lahtinen]
- Expose "mask_area" kwarg. [Panu Lahtinen]
- Remove dataset IDs from the scene info as unecessary. [Panu Lahtinen]
- Fix handling of "delayed" datasets. [Panu Lahtinen]
- Fix topic of message for new files. [Panu Lahtinen]
- Merge pull request 11 from pytroll/add-satpy-sun-check. [Panu
Lahtinen]

Add a check for Sun zenith angle for Satpy plugins
- Merge branch 'add-satpy-sun-check' of
https://github.com/pytroll/trollflow-sat into add-satpy-sun-check.
[Panu Lahtinen]
- Add check for solar zenith angles, don't create composites outside
their range. [Panu Lahtinen]
- Add check for solar zenith angles, don't create composites outside
their range. [Panu Lahtinen]
- Merge pull request 12 from pytroll/feature-satpy-coverage. [Panu
Lahtinen]

Add coverage calculations to SatPy plugins
- Merge branch 'feature-satpy-coverage' of
https://github.com/pytroll/trollflow-sat into feature-satpy-coverage.
[Panu Lahtinen]
- Fix typo: sensor -> 'sensor' [Panu Lahtinen]
- Fix call to Pass() with existing metadata. [Panu Lahtinen]
- Add coverage config item. [Panu Lahtinen]
- Add coverage check. [Panu Lahtinen]
- Move covers() to utils.py. [Panu Lahtinen]
- Fix typo: sensor -> 'sensor' [Panu Lahtinen]
- Fix call to Pass() with existing metadata. [Panu Lahtinen]
- Add coverage config item. [Panu Lahtinen]
- Add coverage check. [Panu Lahtinen]
- Move covers() to utils.py. [Panu Lahtinen]
- Merge branch 'develop' of https://github.com/pytroll/trollflow-sat
into develop. [Panu Lahtinen]
- Fix area missing in some datasets for satpy_writer. [Martin Raspaud]
- Fix info -> attrs rename in satpy_writer. [Martin Raspaud]
- Merge branch 'develop' of https://github.com/pytroll/trollflow-sat
into develop. [Panu Lahtinen]
- Fix .attrs compatibility in satpy resampler. [Martin Raspaud]
- Add .attrs to possible metadata holders for satpy scene. [Martin
Raspaud]
- Remove metadata from Scene instantiation. [Martin Raspaud]
- Update changelog. [Panu Lahtinen]
- Merge pull request 6 from ch-k/feature-publish-vars-param. [Panu
Lahtinen]

Satpy writer parameter to specify published values
- Added sample to template. [Christian Kliche]
- Configuration option to publish everything. [Christian Kliche]

Configuration of satpy_writer now supports "*" in parameter
"publish_vars".

Example 1:

publish_vars: "*"

Example 2:

publish_vars:
"*": ""
super_param: gatherer_time

- Satpy writer parameter to specify published values. [Christian Kliche]

By default writer publishes only a fixed set of variables
in its posttroll message. If you want to forward attributes
that were received from a previous processing stage, you
can define a map called publish_vars. The keys denote variable
names in the message to be published. The value defines the
variable name in the received message.

publish_vars:
gatherer_time: gatherer_time

see example examples/flow_processor_satpy.yaml_template

- Merge pull request 5 from ch-k/feature-param-proj-cache-dir. [Panu
Lahtinen]

Parameter cache_dir for satpy resampler
- Parameter cache_dir for satpy resampler. [Christian Kliche]
- Merge pull request 4 from ch-k/fix-satpy-resampler-radius. [Panu
Lahtinen]

Fix config of resampling radius in satpy_resampler
- Reset 'radius_of_influence' at loop start. [Christian Kliche]
- Fix config of resampling radius in satpy_resampler. [Christian Kliche]
- Merge pull request 3 from ch-k/fix-non-xarray-dataset-attr. [Panu
Lahtinen]

Fix compatibility with satpy non-xarray branch
- Fix compatibility with satpy non-xarray branch. [Christian Kliche]
- Merge pull request 2 from ch-k/fix-writer-restart-params. [Panu
Lahtinen]

Fix writer restart with parameters
- Fix writer restart with parameters. [Christian Kliche]
- Merge pull request 1 from ch-k/feature-scene-reader-param. [Panu
Lahtinen]

Use metadata reader param for scene creation
- Use metadata reader param for scene creation. [Christian Kliche]
- Use the main logger from the fetch file. [Martin Raspaud]
- Change setup.cfg's provides to reflect rpm name. [Martin Raspaud]
- Adapt satpy_writer to xarray branch. [Martin Raspaud]
- Check if file is local before fetching. [Martin Raspaud]

-------------------

Fix
~~~
- Bugfix: use start_time instead of time_slot in satpy_writer. [Martin
Raspaud]

Other
~~~~~
- Update changelog. [Panu Lahtinen]
- Bump version: 0.7.0 → 0.8.0. [Panu Lahtinen]
- Bugfix satpy resampler. [Martin Raspaud]
- Fix satpy resampler for satpy syntax. [Martin Raspaud]
- Bugfix in satpy compositor. [Martin Raspaud]
- Add fetch plugin. [Martin Raspaud]
- Fix PyYAML case as dependency in setup.cfg. [Martin Raspaud]

-------------------
- Update changelog. [Panu Lahtinen]
- Bump version: 0.6.0 → 0.7.0. [Panu Lahtinen]
- Add restart() and is_alive() [Panu Lahtinen]
- Add restart() and is_alive(), remove double setting of logger. [Panu
Lahtinen]
- Move _prev_lock to class attribute, add is_alive() [Panu Lahtinen]
- Move _prev_lock to class attribute, add self.is_alive() [Panu
Lahtinen]
- Skip coverage calculation if min_coverage is not defined. [Panu
Lahtinen]