Directord

Faster client and server interactions and a promoted datastore option allowing Directord to run faster with even fewer resources.


Overview

The new disc datastore, now default in Directord, allows Directord to resume operations faster from a stopped state, it also allows Directord store jobs persistently without consuming system memory or requiring a remote datastore. Another incredible feature of the new datastore is speed. Directord is now able to store, reference, and recall information faster than ever before; this is especially true when dealing with tens of thousands of jobs and orchestrations.

CHANGES

63ffc6d remove artificial time blocks
9cc55b5 Add disc backed data caching for the server

Bug fixes, ennoblement of promoted components, and performance enhancements.


1f58c2b use service component when restarting services (180)
a773663 Add a queue sentinel component (179)
10396ec add force-lock option (178)
f8101de Update query wait to account for null args (177)
51a3c2c add catalog to assist in patching development clients (176)
f7eb207 ensure we're only blocking for tasks that have ended or failed (175)
71f3ed3 add timestamps to job and component returns (174)
4affefb move component lock to a global (173)
c6ca1ea add info when processing workdir (172)
8ca5ab1 faster server processing (169)
b8d2274 add more accurate time data (168)
a249494 add syntax highlighting to readme (166)
9d394c3 ensure that the stdout-arg target is defined (165)

Directord is now more capable, efficient, and **faster** that ever before.


----

Directord is able to connect and operate in FIPS enabled environments.
* libssh2 in our bootstrap process was removed in-favor of libssh.

The user experienced has also been overhauled to provide more useful information at a glance.
* Improved manage commands
* easier to understand logging
* faster information access, even when sorting hundreds of thousands of tasks

In terms of operations, client side logging has been made more robust.
* You can now enable debug and understand most all interactions and follow a process linearly.

Client side execution has also been made massively more efficient.
* No more managers
* Simplified threading
* Intelligent class based locking.

A lot more testing. With the implementation of ASYNC we've added tests to ensure systems work as expected
* Async query tests
* Async race condition tests
* Async caching tests

The comparative analysis tests have also been updated to now show orchestrations with `async` enabled.

Directord now supports asynchronous orchestrations, more components, component threading, updated documentation, and more.


Lots of fantastic changes have gone into Directord, which have all been precipitated by user feedback and increased usage. The project is better tested than ever before and making huge moves toward a rock-solid 1.0 release. While the primary reason for this release is the restructured client code, which enabled asynchronous orchestrations, the fact is this release is jam packed full of exciting features and boring stability improvements.

Many significant improvements have been implemented within Directord.

* Components, drivers, and data-stores are all modular
* Tests have been significantly improved
* Directord now builds and ships RPM packages
* UX improvements have been made
* Fingerprinting now uses SHA256
* Paramiko has been removed in-favor of [ssh2-python](https://github.com/ParallelSSH/ssh2-python); this change alone ensures we're meeting and exceeding our security commitment, even if SSH based bootstrapping is optional.
* Cache file semantics have been improved and documented
* Prod scripts will now install using RPM when running on an RPM based system
* Dev scripts more easily install Directord in debug mode for simpler development.

and more...


Directord now supports data-stores. With this release, the data storage process was moved into a plugin, making it possible to support multiple data-stores for persistence when needed.


While Directord was built to be ephemeral, multiple requests have been made to allow Directord to leverage an external data-store. With this release, the capability to use external data-stores has been developed, with the first external plugin being Redis; additional plugins are planned for future releases.

* The new CLI and configuration option `datastore` has been added using an RFC-1738 compatible string.

Additional changes have been made to clean up code, improve performance, and further enhance the user-experience.