Mrack

Feature

* feat: Retry provisioning strategy

Optional (currently aplied only for OpenStack) provisioning strategy
to retry provisioning of host which ended with error in a provisioning
attempt.

This is useful if environment gets into state where the provisioning
is a bit unstable but in general working. I.e. succeeds for some host
and fails for others.

The Retry strategy:
1. tries to provision all hosts
2. if some failed, delete them and retry only for these hosts
3. if they fail more than max_attempts time(default: 5), fail the job

Default strategy:
- do 1.
- if some failed, fail the job

In each job failure case:
- delete all hosts

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`3683d3c`](https://github.com/neoave/mrack/commit/3683d3c20f917f57de7f88d0b3e9e4fea5ff0398))

Fix

* fix: openstack: log which server is being provisioned

To not see something more useful than:
&34;&34;&34;
OpenStack: Creating server
OpenStack: Creating server
&34;&34;&34;

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`d89aa5d`](https://github.com/neoave/mrack/commit/d89aa5d20192c75560f7bb0beffb3027b0431a47))

* fix: pytest-multihost: handle unresolvable IP into DNS

If host IP was not resolvable into DNS record, external_hostname was
null and thus pytest-multihost could not connect to the host.

This resolves the issue with using IP as a backup.

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`56c716e`](https://github.com/neoave/mrack/commit/56c716ec910028604e2ecaf42124e7e1f6061324))

* fix: Fix making ssh_key_filename absolute for default behavior.

The logic was not working as the comment said. The cause was that
`mhcfg` already contained the value from provisioning config. So
we need to check the metadata to detect the situation.

Do the change also pro-actively for Ansible inventory output.

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`6b5ea95`](https://github.com/neoave/mrack/commit/6b5ea951f132336fcde50b53a97c7b4338eb0099))

* fix: Remove double status translation in parse_errors

parse_errors works with Host objects which already have the normalized
status so usage of STATUS_MAP is incorrect.

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`e5da9cd`](https://github.com/neoave/mrack/commit/e5da9cd6403d67f75d429b3207e11555dd90e437))

Unknown

* 0.5.0

Automatically generated by python-semantic-release ([`211d24e`](https://github.com/neoave/mrack/commit/211d24e9865fa0e2ae468a09cb2ba8be370ae89a))

Chore

* chore: Add manual release action

In this action:
- pre-commit hook is run
- tox is run for a master
After these check succeedes:
- a new commit with changed version and changelog is pushed to master
- a new tag is added to the commit above with particular release version
- a new mrack package is being release to PyPi
(prerequisite: secret with PYPI_TOKEN - already set)

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`01ea068`](https://github.com/neoave/mrack/commit/01ea068c51fe59738644b51f0deb2527f31f2990))

* chore: Add check-commits action for commits

Checks the future commit headers to be compatible with
the project python-semantic-release.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`1716abf`](https://github.com/neoave/mrack/commit/1716abf8aacd9cbdb32fb48531bf056b4d4db57e))

* chore: Add pre-commit github action

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`ace1da3`](https://github.com/neoave/mrack/commit/ace1da3555e568500f32535aabca4aebdb192447))

* chore: Add mypy to the .pre-commit-config.yaml

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`6e3319f`](https://github.com/neoave/mrack/commit/6e3319faf2d577928493243e532168cdea6848d6))

* chore: Add pre-commit configuration files

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`7095ee0`](https://github.com/neoave/mrack/commit/7095ee08640a07a5139fba0270cecc3dc8d60679))

Feature

* feat: Output file paths configurable in mrack config

Adding a possibility to change Ansible inventory and pytest multihost
configuration output file paths in a mrack config file. So that a user
can use directly a custom location.

Example of such config:

ini
[mrack]
mrackdb = config/mrackdb.json
provisioning-config = config/provisioning-config.yaml
metadata = config/metadata.yaml
ansible-inventory = config/test.inventory.yaml
pytest-multihost = config/pytest-multihost-config.yaml


Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`1761baf`](https://github.com/neoave/mrack/commit/1761baf1da0c75e8bb86ec423e3b563bdfb18871))

* feat: Simple way to add hosts into /etc/hosts file

This is useful for cases where the fake hostname needs to be used
e.g. when accessing FreeIPA Web UI which is sensitive on used hostname.

It is uncomfortable task to read the values from `mrack list` and add
them manualy to /etc/hosts.

The change introduces two commands to help with this case:

* `$ mrack eh add` to add all active hosts into /etc/hosts
* `$ mrack eh clear` to remove all records added by mrack.

Mrack adds two markers into the /etc/hosts file and adds records
between them. This should work also with multiple runs of mrack in
separate tests. If the section already contains the fake hostname then
this line is overwritten with a new value. This is useful when
provisioning hosts for the same task after they were previously
removed.

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`da24ac5`](https://github.com/neoave/mrack/commit/da24ac55a3189512c98884eb30f1b5a6ad4dd46a))

Fix

* fix: Use meta_ prefix for parent domain

When inventory was created the expected key value
for the parent_domain in host section should be
meta_parent_domain playbook will not fail with:
The error was: &39;meta_parent_domain&39; is undefined

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`aaeeb58`](https://github.com/neoave/mrack/commit/aaeeb58c0682dbd7be9db94dcb7469c07e6243eb))

* fix: Add missing annotations for mypy

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`2427b15`](https://github.com/neoave/mrack/commit/2427b15ddd12b4d7813a51b3b276b8affd42259b))

* fix: Remove deprecated flag --recursive from Makefile

Remove deprecated flag --recursive for the isort
Add command to sort setup.py.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`48937f3`](https://github.com/neoave/mrack/commit/48937f38a3b8e2780d39539c513187591b9a953d))

* fix: Move common methods to transfromer.py

Move common methods to transfromer.py
to have less copy pasted code.
Added more debug messages to transformation procedure.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`4ea839e`](https://github.com/neoave/mrack/commit/4ea839e560a3d11662d2b65309993652b4cd44bd))

Refactor

* refactor: Add Status badges for PyPI and docs

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`2f5e25a`](https://github.com/neoave/mrack/commit/2f5e25a8d6976c763dc0d31f91682595e2ed212c))

* refactor: Refactor setup.py and configure release

Refactor setup.py to use version from src/mrack/__init__
Add configuration values for python semantic release
to the pyproject.toml and set default branch for releases.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`544baf6`](https://github.com/neoave/mrack/commit/544baf67afff1f91be99ff2167915d901501d9ad))

* refactor: Update README.md file

Update README.md file with sections:
where example inventory output is shown
where usage of the mrack is explained

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`127dd03`](https://github.com/neoave/mrack/commit/127dd031068215673196839a85b5260f25774a9a))

* refactor: Update files after make format

Previous commit which updated .isort.cfg file
would cause failures in testing with black
and isort afterwards.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`6ba0bec`](https://github.com/neoave/mrack/commit/6ba0bec908acadb1085d339e881cf97f954e6832))

* refactor: Fixing the pre-commit eof-check failures

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`93ab0ac`](https://github.com/neoave/mrack/commit/93ab0ac6db0a4091e4f8c83763204d2213a6c44c))

* refactor: Update yamls that yamllint was complaining about

Update yamls that yamllint was complaining about
Update python version to 3.9

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`3c08ec4`](https://github.com/neoave/mrack/commit/3c08ec454a3593aaf2e3f69f89f20fc1481b8c62))

* refactor: Add dsp_name to logging messages to display context

Add dsp_name to logging messages to display context
for prioviders and transformers.
Added some debug messages.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`4c13b6f`](https://github.com/neoave/mrack/commit/4c13b6f5d89006f7886182fe2c83dbdecdf1b585))

Unknown

* 0.4.0

Automatically generated by python-semantic-release ([`4d359ac`](https://github.com/neoave/mrack/commit/4d359acb40aa2852942966140262a6fc1cd2a4e0))

* MrackConfig: alternative approach

This commit implements a MrackConfig which allows to define options
and arguments otherwise needed in a configuration file and thus
save people&39;s time with writing them in CLI.

Config file looks like:

[mrack]
mrackdb = config/mrackdb.yaml
provisioning-config = config/provisioning-config.yaml
metadata = config/metadata.yaml


Where each value is optional and existance of the config is also
optional.

Mrack looks for the config on these locations:
- user provided path via -c or --mrack-config global option
- ./mrack.conf
- ~/.mrack/mrack.conf
- /etc/mrack/mrack.conf

It also changes CLI of commands:
- metadata is no longer an argument but an option: -m or --metadata
- provisioning config shortcut changes to global -p

With these changes it is possible to define a config file to allow
just typing:

$ mrack up
$ mrack list
$ mrack ssh
$ mrack output
$ mrack destroy


The MrackConfig class is also extensible for new use cases.

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`84e52a2`](https://github.com/neoave/mrack/commit/84e52a29090a66d5ba06c9ea8d7a5de48001f4c9))

* Add mrack.conf from which we load defaults

Add mrack.conf which contains default enfironment
configuration in ini format for the click options
- `--config=provisioning-config from mrack.conf`
- `--db=mrackdb from mrack.conf`

Load default for options from the mrack.conf file from:
1. use `$(pwd)/mrack.conf` or other way the `./mrack.conf` - actual directory
2. or use user home directory configuration - `~/.mrack/mrack.conf`
3. alternatively directory `/etc/mrack/mrack.conf` as safe fallback.

Add example `mrack.conf` and `provisioning-config.yaml`
to the `data/` directory in repository.

Update setup.py to install required files from `data/` dir. ([`951c08f`](https://github.com/neoave/mrack/commit/951c08fb9e80c3dc099751e08f6ed16594c7362d))

* Update .gitignore for python projects

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`8f9a674`](https://github.com/neoave/mrack/commit/8f9a67435304497132b0f0cc1d0a2c17e76cc4e7))

* Support netbios in windows host definition for the inventory

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`1916058`](https://github.com/neoave/mrack/commit/1916058f57aea85da42f34b4f300866c228403ed))

* Give preference to metadata ssh_key_filename in pytest multihost

If job metadata contains a ssh_key_filename, use it for pytest
multihost configuration. I.e. do no override with key used by mrack.

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`a1cad10`](https://github.com/neoave/mrack/commit/a1cad10b33034a8244d1f15fec54e56e96d257f3))

* Do not set username for pytest multihost config

We have actually not used it at it cannot be assumed that
the user used for provisioning should be used for pytest-multihost.

E.g. a lot of tests rely on root - to be able to completely control
the remote machine. That cannot be done, e.g. with default cloud-user.

(Though it can often sudo to root)

Let&39;s keep there the more simple configuration, where user is the
same for all host and is defined in provisioning config &39;mhcfg&39; dir.

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`7bb230e`](https://github.com/neoave/mrack/commit/7bb230e170ac0a2373a2316ef23a26bfcb681ad9))

* Fix OpenStack provisioning status translation

It is not needed to use a STATUS_MAP for translation status in
prov_result_to_host_data as it is done also in to_host of parent
class. Double translation doesn&39;t work and thus everything provisioned
had incorrecly status : &34;other&34;.

This has then a negative effect on ssh actione where no host is reported
as active.

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`a9d62cc`](https://github.com/neoave/mrack/commit/a9d62ccd9206f594dc4b2abec1020a22607980a2))

* Do not show traceback on ApplicationError

So that, e.g., on ssh action, when there are no active host, user
will not get an exception ending with:


raise ApplicationError(&34;No active host available.&34;)


But only:

No active host available.


Which is more user friendly.

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`99c5ab1`](https://github.com/neoave/mrack/commit/99c5ab1ca747da35576a6300cd54a0cd1230d867))

* Increase OpenStack timeout to 6mins

Trying to make the provider more stable if OpenStack is slow, e.g.
under load.

It&39;s not very clever method. But might work.

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`7854c49`](https://github.com/neoave/mrack/commit/7854c4937742ec5ee0cbe05370c324061b3ebdfd))

New features:

Create CODE_OF_CONDUCT.md
Add basic image validation to AWS and Beaker
Support restraint_id record in metadata
Add docs on how to set credentials
Build and publish html to Azure artifacts storage

Bug Fixing and testing:

docs: set master doc to index
Remove missing and unused print_basic_info method
Fix destroy action because delete host now needs only host.id
Require specific versions for mrack
Fix regression in up action with OpenStack provider
Remove unused values in aws and beaker tranformation

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`145b50a`](https://github.com/neoave/mrack/commit/145b50a1a67b92cfff910ea633cd7b3f8ea4ad37))

* Remove unused values in aws and beaker tranformation

These values are loaded from metadata when generating
inventory so they do not need to be passed to provider.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`6b379e1`](https://github.com/neoave/mrack/commit/6b379e1e83512b42ffaebd979d52238bd766ca90))

* Fix regression in up action with OpenStack provider

Provisioning failed because requirements contained unexpected information.

I.e., the provider should receive only the information it expects or
are valid for OpenStack provisioning.

Removing the culprit lines: meta_image and restraint_id params.

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`4cdedd1`](https://github.com/neoave/mrack/commit/4cdedd1d49850eee4229a84c36e1fdf45684be29))

* Require specific versions for mrack

It is always better to require specific versions
of dependencies because pip always installs
the latest and they may break funcionality
or create conflicts when mrack is a dependency.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`2c20866`](https://github.com/neoave/mrack/commit/2c2086664418120a521b03c9a6237bb09fe01d0d))

* Build and publish html to Azure artifacts storage

Signed-off-by: Armando Neto <abiagionredhat.com> ([`305ebc2`](https://github.com/neoave/mrack/commit/305ebc2a55f2a708879859518a2ad410169bbf2b))

* Add docs on how to set credentials

Quick guide on how set credentials for current supported providers.

Signed-off-by: Armando Neto <abiagionredhat.com> ([`ef46be9`](https://github.com/neoave/mrack/commit/ef46be9b8411312a9b333abe380411ed21b216fa))

* Fix destroy action because delete host now needs only host.id

Because of delete_host method now needs only host.id,
the Destroy action was broken and it needed to use
the host.id variable not whole object as well.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`abbb80c`](https://github.com/neoave/mrack/commit/abbb80c930c2acb519d583bb82efda8abee55f89))

* Support restraint_id record in metadata

Support restraint_id defined in metadata and later
defined in inventory for the restraint jobs.
For this we pass requirements all the way
for later inventory generation.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`59aa87f`](https://github.com/neoave/mrack/commit/59aa87f97e83c2784f0fe46da1fd22dc61f4f917))

* Remove missing and unused print_basic_info method

Add loger info message to error parsing method.
Move the parse_errors to provider.py file.
Add property error to Host object.
Delete host now needs only host_id.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`c6f8394`](https://github.com/neoave/mrack/commit/c6f8394bdfe421f9275dfb4bd2b802e9edfc9b23))

* Add basic image validation to AWS and Beaker

Added basic image validation to AWS and Beaker provider
When os was not known in provisioning config, null/None
was taken and set as required image even for Openstack.

Value of &39;os&39; is passed to req when image is not in place.
Then the &39;os&39; value is seen in ValidationError messages
even instead of None/null in Openstack, AWS and Beaker.

Support image/distro definition from metadata.
And flag it with meta_distro/meta_image when used.

For AWS check image availability with boto.
When image is not present raise ValidationError

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`1e3b298`](https://github.com/neoave/mrack/commit/1e3b298b26a7e6f494a7c9c9adfcd6a17ff1abd3))

* Create CODE_OF_CONDUCT.md

https://github.com/neoave/mrack/issues/39 ([`38a7a74`](https://github.com/neoave/mrack/commit/38a7a746c7e7be8cda93faeab1aebac02aa320be))

Bug fixing and testing:

Refactor providers to share same code
Force black version to get rid of test errors
Remove false warnings produced by isort
Add missing beaker job status
ansible-inventory: rework generation of groups

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`e5bc921`](https://github.com/neoave/mrack/commit/e5bc9211bcbfc8a8fe1709186e21c337e20fd55e))

* Force black version to get rid of test errors

When black devels change their opinion too much
we got format errors. Let us try to use versioning here.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`38b3f07`](https://github.com/neoave/mrack/commit/38b3f07292624efb55d3ac55caeb1dca0b4c2496))

* Refactor providers to share same code

Methods provision_hosts, delete_hosts and to_host
were moved to provider.py as they can be easily
shared among the providers.

Added missing methods to provider.py to share
same structure. Not implemented methods will throw
a NotImplementedError() and they are ment to be
implmented per provider.

Removed set poll sleeps based on host count.

For all providers moved STATUS_MAP to class
so from now on it is a class attribute.

Openstack&39;s wait_till_provisioned now uses
instance and not only id to match inherited method.

Added optional username parameter to to_host method in provider.py

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`b0254b2`](https://github.com/neoave/mrack/commit/b0254b2056fd05b742ba544f167f01cd1b464c89))

* ansible-inventory: rework generation of groups

The Ansible inventory output was not correctly adding hosts into groups. E.g.
group was not added when it was not defined in layout. Nested group was not
correctly found and host added.

This part is completely reworked and changes the behavior to:
- add only a host name reference (no vars) into custom groups
- add missing groups as children of &39;all&39; group
- works with both &34;groups&34; and &34;group&34; defined in metadata
- add all hosts to &39;all&39; with also variables. This ensures that the full host
object is defined only once.

Doc: https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`23cdb21`](https://github.com/neoave/mrack/commit/23cdb21996310b35ea200cba36cec21d41839526))

* Add missing beaker job status

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`f476a12`](https://github.com/neoave/mrack/commit/f476a1277fe9290b380e7acecf63e01d3526b772))

* Remove false warnings produced by isort in make format

Signed-off-by: Petr Vobornik <pvoborniredhat.com> ([`10bd910`](https://github.com/neoave/mrack/commit/10bd910bdac47f08af4b45ab613ec52f300028ab))

Bug fixing and testing:

Fixed username for beaker machines to be root.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`364ab4c`](https://github.com/neoave/mrack/commit/364ab4cb571eb97c609bdaccadc634f3415f02b2))

* Add username=&39;root&39; to fix ansible user

Added username=&39;root&39; to beaker host object
initialization so inventory is created
using ansible_user: root

Fix initialization of provider in a way
that we do not pass provisioning config
but only needed values.

Add status Queued as provisioning.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`b25a173`](https://github.com/neoave/mrack/commit/b25a173a02474be644b51df513640d64536f425c))

New features:

Beaker provider support

Bug fixing and testing:

Use public AWS addresses
Remove asserts and replace them with ProvisioningErrors

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`02a9fb6`](https://github.com/neoave/mrack/commit/02a9fb62accea530ac658ec6a23b88f7d126e6f2))

* Use public AWS addresses instead of private ones

Using private addresses will result into unreachable host
without vpn tunnel or mechanism to access AWS network.

Add security group, tags, keypair to init of AWS provider.
They have to be defined in provisioning config.

Remove asserts and replace them with ProvisioningErrors

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`79aa885`](https://github.com/neoave/mrack/commit/79aa8857995fa51599e9adf61c047efc286685a1))

* Add beaker provider

Beaker provider generates jobxml and then schedules
this job to beaker hub. Provider waits until job
is in Reserved state which is success.

Beaker provider requires kerberos credentials.

Signed-off-by: Tibor Dudlák <tdudlakredhat.com> ([`40dcb7f`](https://github.com/neoave/mrack/commit/40dcb7ff972408da74821d63fec92846227d65a2))