Ksconf

~~~~~~~~~~~~~~~~~~~~~~~~~~

- **CLI change**: Replaced short option for ``--allowlist`` to be ``-a``, before it was ``-w``.
I assume this was left over early development where the argument was initial called ``--whitelist``, but at this point ``-w`` is just confusing.
Normally, I'd keep ``-w`` for a period of time and issue a deprecation warning.
However, given that 0.8.0 was released less than a week ago, and that ksconf package is an "alpha" feature,
I'm going to make this change without prior warning.
- Add some safety checks to the package command to check for app naming issues (where the app folder doesn't match ``[package] id`` value in ``app.conf``), and hidden files and directories.
- Add new ``{{app_id}}`` variable that's usable with the ksconf package command.
- Added a new optional argument to :py:func:`~ksconf.builder.steps.copy_files` called ``target`` for additional control over the destination path of artifacts copied into the build folder.
- Minor tweak to unhandled exceptions. The name of the exception class is now show, and may be helpful in some situations.
- When using ``make_missing`` in :py:class:`~ksconf.conf.parser.update_conf`, missing directories will now be created too.
- Additional fixes to the Ksconf for Splunk App ``build.py`` script: Now explicitly creating a top-level ``ksconf`` folder.
It's likely that this was the root cause of several other issues.

~~~~~~~~~~~~~~~~~~~~~~~~~~

- Fixed bugs created by v0.8.2 (yanked on pypi)
- Properly resolved issues with Splunk app building process.
- Open issue uncovered where ``ksconf package`` can produce a tarball that's unusable by Splunkbase.

~~~~~~~~~~~~~~~~~~~~~~~~~~

- Fixed some build issues with the Splunk app. (The splunk app is now built with ``ksconf package`` and the ``ksconf.builder``)
- Minor doc fix up; you know, the stuff typically found minutes after any new release :-)

----------

**Highlights:**

* New command :ref:`ksconf_cmd_package` is designed for both Splunk developers and admins * New module :py:mod:`ksconf.builder` helps build Splunk apps using a pipeline; or when external Python libraries are bundled into an app
* Legit layer support with built-in layer filtering capabilities is available in several commands
* Python 3! Head's up: We'll be dropping support for Python 2 in an upcoming release

.. note::

Come chat about ksconf on `GitHub discussions <https://github.com/Kintyre/ksconf/discussions>`__ even if it's to say we should use some other forum to stay in touch.

**What's new:**

- The **new ksconf package command** supports the creation of Splunk app ``.spl`` files from a source directory.
The ``package`` command can be used by admins to transfer apps around an organization, while keeping the ``local`` folder intact,
or by a developer who wants ``local`` to be automatically merged into ``default``.
The app version can be set based on the latest git tag by simply saying ``--set-version={{git_tag}}``.
- The **ksconf.builder Python module** is a API-only first for ksconf!
This build library allow caching of expensive deterministic build operations, and has out-of-the-box support for frequent build steps like adding Python modules locally using ``pip``.
As the first feature with no CLI support, I'm exceeded to get input from the broader community on this approach.
Of course this is just an experimental first release.
As always, feedback welcome!
- **Native support for layers!**
It's official, layers are now a proper ksconf feature, not just an abstract concept that you could throw together yourself given enough time and effort.
This does mean that ksconf has to be more opinionated, but the design supports switching layer methods,
which can be extended over time to support new different strategies as they emerge and are embraced by the community.
Supports layers filtering as a native feature. This has always been technically possible, but awkward to implement yourself.
Layer support is currently available in :ref:`ksconf_cmd_combine` and :ref:`ksconf_cmd_package` commands.
- **Moving to Python 3 soon.**
In preparation for the move to Python 3, I've added additional backport libraries to be installed when running Python 2.
Support for Python 2 will be dropped in a future release, and anyone still on Splunk 7 who can't get a Python 3 environment will have to use an older version of ksconf.
Also note that when jumping to Python 3, we will likely be requiring Python 3.6 or newer right out of the gate. (This means dropping Python 2.7, 3.4 and 3.5 all at the same time.)
Whoohoo for f-strings!
- **CLI option abbreviation has been disabled.**
This could be a breaking change for existing scripts.
Hopefully no one was relying on this already, but in order to prevent long-term CLI consistency issues as new CLI arguments are added, this feature has been disabled for all version of Python.
This feature is only available, and was enabled by default, starting in Python 3.5.
- **Removed insensitive language.**
Specifically the terms 'whitelist' and 'blacklist' have been replaced, where possible.
Fortunately, these terms were not used in any CLI arguments, so there should be no user-facing changes as a result of this.
- **Removed support for building a standalone executable (zipapp).**
This packaging option was added in v0.4.3, and deprecated in v0.6.0 once the Splunk app install option became available.
I'm pretty sure this won't be missed.


**API Changes**

- NEW API :py:mod:`ksconf.builder`
The documentation for this module needs work, and the whole API should be considered quite experimental.
The easiest way to get started is to look at the :doc:`Build Example <build_example>`.

- NEW Context manager :py:class:`~ksconf.conf.parser.update_conf`.
This enables super easy conf editing in Python with just a few lines of code.
See docs API docs for a usage example.

**Developer changes:**

- Formatting via autopep8 and isort (enforced by pre-commit)
- Better flake8 integration for bulk checking (run via: ``tox -e flake8,flake8-unittest``)

~~~~~~~~~~~~~~~~~~~~~~~~~~

In addition to the 0.8 summary above, 0.8.0 specifically includes the following changes:

- Add automatic layer support.
Currently the two supported layer schemes are (1) explicit layers (really this will ``disable`` automatic layer detection), and (2) the ``dir.d`` format which uses the ``default.d/-layer-name`` style directory support, which we previously promoted in the docs, but never really *fully* supported in a native way.
This new ``dir.d`` directory layout support also allows for multiple ``*.d`` folders in a single tree (so not just ``default.d``), and if your apps have different layer-points in different apps, it's all handled transparently.
- Layer selection support was added to the ``combine`` command.
This allows you to ``--include`` and ``--exclude`` layers as you see fit.
See the docs for more details and examples of this new functionality.
This works for both the new ``dir.d`` directories and the explicit layers, though moving to the ``dir.d`` format is highly encouraged.
- New cheatsheet example: Using ``ksconf package`` and ``splunk install app`` together.
- Updated the combine behavior to optimize for the situation where there is only a single conf input file provided.
This behavior leaves any ``.conf`` or ``.meta`` file untouched so there's no sorting/normalizing or banner.
See `64 <https://github.com/Kintyre/ksconf/issues/64>`__.
- Eliminated an "unknown command" error when one of the ksconf python modules has a SyntaxError.
The new behavior isn't perfect (you may still see "unrecognized arguments"), but overall it's still a step in the right direction.



Ksconf 0.7.x
------------

New functionality, massive documentation improvements, metadata support, and Splunk app install fixes.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Fixed bug where empty stanzas in the local file could result in deletion in default with ``ksconf promote``.
Updated diff interface to improve handling of empty stanzas, but wider support is still needed across other commands; but this isn't a high priority.