Luigi

Added:

luigi
* Adds "Force Commit" button in UI to set tasks to DONE 2751
* Show task history link in visualizer when recording. 2759

Fixed:

luigi

* Replace documentation reference to outdated test environment py27-nonhdfs 2762
* Issue 2644: Tasks can be run several times under certain conditions 2645

luigi & luigi.contrib
* Ensure ignored tests are picked up by tox 2758

Changed:

luigi

* Update tornado requirement for new enough python versions 2761

luigi.contrib

* contrib/ftp: Clean up temporary files 2755

Added:

luigi
* Expandable Namespace Folders for the Visualiser Sidebar 2716
* Added new companies to the luigi users list: 2730 2747

luigi.contrib
* Enable Overriding Poll Interval for Kubernetes Jobs 2724
Fixed:

luigi
* Code example correction 2754

Changed:
luigi
* Update release process 2727
* Change GET request to POST requests in luigi/rpc 2732
* Fix SendGrid email API documentation. 2745

Added:

luigi
* Add `check_complete_on_run` to optionally mark tasks as failed if complete() is false when run finishes (2710)
* Add section "Running Luigi on Windows" to docs (2720)
* Add Giphy to list of companies using Luigi to docs (2713)

luigi.contrib
* Add DropboxTarget for luigi (2696)

Fixed:

luigi
* UI: Fix time graph - y axis to account for timezones. (2711)

Changed:
luigi
* Bump dependencies used by SendGrid integration. (2715)
* Update copyright year in LICENSE (2723)

luigi.contrib
* Make RedisTarget compatible with redis-py >= 3 (2722)

Fixed:

luigi
- fix calling method on wrong object 2709
This fixes 2704.

Fixed:

luigi
- (Doc) Fix example of "summary_length" 2700
- Fix `__init__` error when using TOML config 2702
- add callback to metric collector 2704

luigi.contrib
- Fix BigQueryTarget parsing in beam_dataflow module 2705

Changed:

luigi.contrib
- aws batch : job queue as parameter 2689

This release is broken due to 2628 .

Added:

luigi
- Added support for a detailed LuigiRunResult instead of a plain Boolean (2630)
- Add worker option 'max_keep_alive_idle_duration (2654)
- Added worker-id commandline parameter (2655)

luigi.contrib
- Add support for specifying kubernetes namespace (2629)
- Add a Task wrapper for MicroSoft OpenPAI (2531)
- Provide automatic URL tracking for Spark applications (2661) (2669)
- add beam_dataflow_task to luigi/contrib (2675)

Both
- Add Prometheus contrib for monitoring purpose (2628)

Fixed:

luigi
- setup.py: Support older setuptools (<=20.1.1) (2623)
- Fix broken aws tests (2658)
- Accept pathlib based path as argument for LocalTarget (2548)
- Fix durations in D3 graph (fixes 2620) (2624)
- Import collections ABCs from collections.abc, not collections (2683)
- Configuration documentation: remove deprecated/wrong [core]max_reschedules entry (2692)

luigi.contrib
- fixes 2223 HdfsTarget is not working with snakebite (2572)
- Add port field for PostgresQuery (fixes 2625) (2627)

Both
- Fix flake errors after moving to python 3 (2695)

Changed:

luigi
- Simplify implementation of temporary_path() (2652)
- Prevent range-tasks from autoloading (2656)
- Replace Python MapReduce example with Spark example. (2668)
- Minor improvements to single-worker-timeout support (2667)
- Require at least python-dateutil version 2.7.5 instead of only 2.7.5 (fixes 2662) (2679)
- RangeMonthly should deal with whole months (2666)
- Reconcile underscore/dash config style handling (2688) 2691

luigi.contrib
- Add autodetect parameter to BigQueryLoadTask (2363) (2575)