Dactyl

v0.7.0 brings several new features, slight changes to default behavior, and some bug fixes and cleanup to Dactyl.

**Breaking Changes**

The default behavior when building HTML is now to copy static files into the output directory. This means the `-s` flag is usually not necessary anymore. If you _do not_ want to copy static files to the output directory, provide the `--no_static` (`-S`) flag instead.

There are also new flags for copying only content static (`-C`) or template static (`-T`) files.

**New Features**

Dactyl can now generate JSON formatted for upload to ElasticSearch using the `--es` flag, or even upload directly to ElasticSearch when building any type of content using the `--es_upload`. It only builds / uploads files that have an `md` source file. You can define custom templates for the JSON output. (See the [v0.7.0 README.md](https://github.com/ripple/dactyl/blob/v0.7.0/README.md) for details.)

**Bug Fixes and Cleanup**

- Filter imports are now compatible with Python 3.4 and lower ( 24 )
- Better error messaging for missing/broken filters and other issues ( 23 )
- Correctly outputs the version number or help statement even if the config file is invalid ( 22 )
- When using `--bypass_errors`, a preprocessing error falls back to simply parsing the Markdown content as-is instead of resulting in a blank page.
- Starts refactoring the source files to split up functionality better into smaller pieces for easier reusability.
- Unifies logging and output controls across `dactyl_build`, `dactyl_link_checker`, and `dactyl_style_checker`
- Repository includes integration tests for the first time

This release changes the error message for the case when multiple targets in the config file use the same `name` value, or a target in the config file omits the required `name` value. Dactyl v0.6.2 also catches an error case that may be missed in prior versions if the number of targets with repeated and missing `name` values offset each other exactly.

The old message was:

Duplicate or missing target name in config file

In Dactyl v0.6.2, the two cases have different error messages.

If a target does not have a `name` field, the error includes the target object that is missing the name:

Target does not have required 'name' field: {'display_name': 'Dactyl Test Suite', 'intro_blurb': 'Dactyl is great! It makes indexes like this automatically!', 'condition': 'tests-2'}

If a target reuses the `name` of a previous target, the error contains the repeated name:

Duplicate target name in config file: 'rs-api-reference'

Unless you specify `--bypass_errors`, Dactyl exits after the first error of either type, even if the config file contains more such errors.

Additionally, v0.6.1 comes with two more consistency checks:

- Checks that all targets have unique `name` values.
- Checks that all targets in each page definition actually exist. (This catches some cases where a page can be omitted from a target because the target name was misspelled.)

As before, Dactyl v0.6.1 exits if these checks fail, whereas previous versions bypassed the errors silently. You can bypass the errors with the `--bypass_errors` commandline option if desired.

In most cases, you can try to build anyway if you pass the `--bypass_errors` commandline option; however, you should probably fix the errors in your config file instead.

Some common errors include:

Invalid targets array

Previous versions of Dactyl mishandled the case where the `targets` array that didn't parse as array syntax in YAML. For example:


- name: Correct Page
md: correct.md
html: correct-page.html
targets:
- mytarget

- name: Broken Page
md: broken.md
html: broken-page.html
targets:
-mytarget
Without the space after the -, the above target list
parses as a string instead of a one-item array


Prior to Dactyl v0.6.0, the broken page would be silently omitted from all targets because its `targets` parameter was present but not formatted as the correct type. Dactyl v0.6.0 exits immediately with an error if it reads a config file containing such an error.

Reused HTML Filename

Previous versions of Dactyl silently overwrote a page if a later page in the same target used the same HTML filename as output. For example:


- name: Losing Page
md: losing.md
html: overlap.html
targets:
- mytarget

- name: Winning Page
md: winning.md
html: overlap.html
targets:
- mytarget


With old versions of Dactyl, the file `overlap.html` ends up with the contents of Winning Page when you build `mytarget`. With v0.6.0, Dactyl exits with an error if you try to build `mytarget` with the above config file.

Default HTML Filenames

Prior to v0.6.0, you had to specify an HTML filename in the `html` field for each page in the Dactyl config. Starting with v0.6.0, the `html` field is optional; if omitted, Dactyl chooses a filename based on the following rules:

1. If the page has an `md` field, Dactyl uses that to make a .html filename. Dactyl replaces `.md` with `.html` and replaces the OS's path separator (typically `/`) with a dash (`-`) instead.
- If you want to output HTML files in folders that mimic the input structure, add `flatten_default_html_paths: true` to the top level of your Dactyl config file. This might not work with PDF output for all versions of Prince on all platforms.
2. If the page doesn't have an `md` field but does have a `name` field, Dactyl uses a "slug" version of the `name`, appending `.html`.
3. If the page doesn't have an `html`, `md`, or `name` field, Dactyl generates a filename from the current [`time()`](https://docs.python.org/2/library/time.html#time.time) value, replacing the `.` with a `-` character. Because this value is different each time you run Dactyl, continuously building the same target results in an ever-growing number of files, so this is not advised.

Fixes a bug in the `link_replacement` filter that caused `image_re_subs` to raise an exception and end the build process.

While still technically a "0." release, v0.4.0 is a major refactor of Dactyl's source code!

As part of the refactor, many unintuitive or weird details have been removed. A few things have been renamed to be more consistent and sensible.

Resolves:
- 1 - specify one or more `filter_paths` in config to be able to import custom filters.
- 2 - The hard-to-explain link substitution filter from the Dactyl core has been removed. The new `link_replacement` filter covers the same functionality in a slightly different way. Instead of link replacements being part of page definitions, they're defined in a `link_subs` map in the target definition.
- 4 See the examples in this repo.
- 15 - The `pdf_template` field at the global level has been renamed `default_pdf_template`. The old name still works but gives a warning.
- 16 - Callouts now use the `dactyl-callout` HTML class by default, but you can change it back or otherwise customize it using the `callout_class` config option. Additionally, you can define custom callout types by adding a `callout_types` array to the config. (The default is still tip/note/caution/warning.)
- 17 - Both the preprocessor and templates get new fields, with the preprocessor getting many more. See the readme for full tables.

Additionally:
- Filters get many more arguments, including access to global config, the Dactyl logger object, and categories. It's recommended that the `filter_markdown()`, `filter_html()`, and `filter_soup()` functions in a filter take a **kwargs argument to maintain compatibility with future versions of Dactyl, which may add more keyword arguments.
- Githubify mode has been removed and replaced by a similar Markdown export mode (`--md`). Markdown mode runs the preprocessor _and_ markdown filters on the input files. It also builds every file by default. ("Githubify" mode didn't run filters, although it attempted link substitution, and it only worked on a single file.)
- New `--only` option allows you to build only a single file from a target. (This is similar to how Githubify used to work, except now it can be used with HTML and PDF too.) Pass the input (`*.md`) or output (`*.html`) filename as the argument.
- Output paths can now contain folders. Dactyl automatically creates folders as necessary.
- New built-in filters: `link_replacement` (replaces the removed link substitution functionality) and `demote_headers` (helps to adjust header levels to fit appropriately in your hierarchy without editing the markdown files directly; can be run only in PDF mode, too)
- The `sidebar_content` template variable has been renamed (more semantically) `page_toc`; the old name remains as an alias. Dactyl no longer sets the `sidebar` field of a page automatically.
- The preprocessor now runs even on markdown files that are remotely sourced via http/https
- The `buttonize` filter now removes the trailing `>` that causes links to be "buttonized"