Keymap-drawer

This release updates ZMK parsing to use a tree-sitter-based parser and adds support for ZMK physical layouts in devicetree format (used by Studio), among other improvements.

The new parsing implementation results in speed improvements that range from 10% shorter runtime to 7x, with a median speed-up of ~2x in my test cases.
If you encounter any new problems with parsing your keymaps, please open an issue!

Features

Drawing

- Add [`dts_layout` option](https://github.com/caksoylar/keymap-drawer/blob/v0.19.0/KEYMAP_SPEC.md#zmk-physical-layout-specification) for physical layouts, and corresponding `--dts-layout` CLI arg.
- This is analogous to `qmk_info_json` field pointing to a file path, except the file should contain ZMK's physical layout nodes in devicetree
- **Deprecated** `qmk_layout` field in physical layout definitions, and corresponding`--qmk-layout` CLI arg.
Use `layout_name` instead, which also works with `dts_layout` above

Parsing

- Add [`parse_config.layer_legend_map`](https://github.com/caksoylar/keymap-drawer/blob/v0.19.0/CONFIGURATION.md#layer_legend_map) which can be used to customize how layer names are represented in the keymap
- This is similar to how e.g. `zmk_keycode_map` is used to change keycode representations, but applied to layer names instead
- Switch to a [`tree-sitter-devicetree`](https://pypi.org/project/tree-sitter-devicetree/)-based parser for ZMK, from the old `pyparsing`-based one
- Add `--virtual-layers` CLI argument to `keymap parse`
- This appends empty "virtual" layers at the end of the keymap; this can be used for assigning combos etc. to show them on a separate diagram
- Also look-up behavior prefixes in `raw_binding_map`
- For example match entries like `&td` for ZMK behavior bindings like `&td 0` to fix for behaviors nested in hold-taps

Fixes

- Ignore "reserved" ZMK layers while parsing, used for ZMK Studio
- Better support for Studio-related ZMK updates to come, see 125
- Switch to using Github raw URL for `mdi` glyphs, by magicDGS in 132
- The old unpkg URL was unreliable and seems to have eventually disappeared
- Add retries and a more generous timeout for fetching SVG glyphs from remote URLs
- More correctly center/normalize QMK physical layouts when there are rotated keys

ZMK drawing workflow

- Use pipx for installs in draw-zmk.yml (PEP 668) by janwvjaarsveld in 118
- Enable extended globbing for pattern matching by thazhemadam in 121

New Contributors
* janwvjaarsveld made their first contribution in 118
* thazhemadam made their first contribution in 121
* dhruvinsh made their first contribution in 126
* magicDGS made their first contribution in 132

**Full Changelog**: https://github.com/caksoylar/keymap-drawer/compare/v0.18.1...v0.19.0

This release contains a couple bug fixes.

- Avoid newlines between 2 tspan elements by wouterj in https://github.com/caksoylar/keymap-drawer/pull/114. This fixes the annoying horizontal mis-alignment when the tap legend had 3+lines.
- Add an additional `u` alias for key width in json-format `qmk_keyboard`. This is sometimes emitted by https://nickcoutsos.github.io/keymap-layout-tools after importing e.g. Kicad PCB files.

This release adds support for parsing ZMK configs that use modules like [zmk-helpers](https://github.com/urob/zmk-helpers/) with additional config options, along with adding special styling to layer key legends and making them clickable.

Edit: `zmk-helpers` now have [docs](https://github.com/urob/zmk-helpers/blob/main/docs/keymap_drawer.md) on using `keymap-drawer` with it using below changes.

Features

Drawing

- **Breaking change:** Add new config option [`draw_config.style_layer_activators`](https://github.com/caksoylar/keymap-drawer/blob/v0.18.0/CONFIGURATION.md#style_layer_activators) which defaults to true. This marks layer activators legends by adding a link that goes to the corresponding layer (might not work for embedded SVGs), and adding an underline by default.
- If you want to restore the old behavior, this setting can be disabled in the config file: `draw_config: { style_layer_activators: false }`
- You can change the styling, e.g. if you want italic instead of underlined:
yaml
draw_config:
svg_extra_style: |
text.layer-activator { text-decoration: none; font-style: italic }

- Add a new CSS class `alternate` so you can define separate styling for extra activators marked by the [`parse_config.mark_alternate_layer_activators` config](https://github.com/caksoylar/keymap-drawer/blob/v0.18.0/CONFIGURATION.md#mark_alternate_layer_activators) (no styling defined by default) (98)
- Example:
yaml
parse_config:
mark_alternate_layer_activators: true
draw_config:
svg_extra_style: |
rect.held.alternate { fill: 478583; }
media (prefers-color-scheme: dark) { rect.held.alternate { fill: 478583; } } some other color here

- Add `hidden` property to the [combo spec](https://github.com/caksoylar/keymap-drawer/blob/v0.18.0/KEYMAP_SPEC.md#combos), so you can avoid drawing certain combos that the parser outputs by adding `hidden: true` (for instance through [`parse_config.zmk_combos`](https://github.com/caksoylar/keymap-drawer/blob/v0.18.0/CONFIGURATION.md#zmk_combos)) (104)
- Example:
yaml
parse_config:
zmk_combos:
combo_esc: { hidden: true }

- Support using underscore instead of space as split separator for the [cols+thumbs notation](https://github.com/caksoylar/keymap-drawer/blob/v0.18.0/KEYMAP_SPEC.md#colsthumbs-notation-specification) spec in order to reduce command line quoting issues
- Example: `keymap draw -n 33333+2_3+233331 my_asymmetric_keymap.yaml`

- Implement support for [`layout_aliases`](https://docs.qmk.fm/reference_info_json#layouts) in `qmk_keyboard` physical layout definitions

Parsing

- Add [`parse_config.zmk_additional_includes`](https://github.com/caksoylar/keymap-drawer/blob/v0.18.0/CONFIGURATION.md#zmk_additional_includes) to augment the preprocessor include path (105)
- This can be used to be able to parse keymaps using `zmk-helpers` using the module functionality, since `include`s aren't using relative paths in that case. Example:
yaml
parse_config:
zmk_additional_includes: [ "zmk-helpers/include" ]

- If you want to use it in the automated ZMK drawing workflow, use the new `west_config_path` argument to the workflow (107) so that the modules defined in the `west.yml` file are automatically fetched. (You still need the above value in the config.) Example:
yaml
jobs:
draw:
uses: caksoylar/keymap-drawer/.github/workflows/draw-zmk.ymlmain
permissions:
contents: write allow workflow to commit to the repo
with:
[...]
west_config_path: 'config' this folder contains west.yml with module definitions

- Module fetching is not (yet) supported in the ZMK URL parse option of the [web app](https://caksoylar.github.io/keymap-drawer/)
- Add [`parse_config.zmk_preamble`](https://github.com/caksoylar/keymap-drawer/blob/v0.18.0/CONFIGURATION.md#zmk_preamble) config option which is text inserted at the beginning of the keymap, usually used to modify preprocessor behavior. It defaults to `define KEYMAP_DRAWER` (103)
- The default value exposes the existing behavior which was being defined internally before this change
- Restore support for using modified keycodes like `LC(LEFT)` in `parse_config.*_keycode_map` that was removed in v0.16 (106)

Fixes

- More robust devicetree parsing because of a few fixes involving parsing phandle arrays, such as when elements can have commas or split to multiple `<...>, <...>;` blocks (102)
- More robust QMK keymap.json parsing, for cases where the keycodes can contain extra spaces

Misc

- Add a physical layout definition for `hummingbird` keyboard
- For unibody keyboards like `fingerpunch/ffkb` and `rufous`, change the default physical layout to avoid all keys being rotated. This should improve readability by sacrificing faithfulness to the actual layout
- The older rotated versions are available using `fingerpunch/ffkb_rotated` and `rufous_rotated`

**Full Changelog**: https://github.com/caksoylar/keymap-drawer/compare/v0.17.0...v0.18.0

This is a release with minor external-facing changes, since most of them were internal tweaks to help downstream users like the web app.
I recently spent some time working on it at https://caksoylar.github.io/keymap-drawer and implemented a few improvements (and added sorely needed tooltips). If you haven't checked it out lately, you might want to have a look!

Also if you are a Kanata user, you can check out the work-in-progress Kanata config parser at https://github.com/caksoylar/keymap-drawer/tree/kanata and related issue #95.

Features

Drawing

- **Breaking change**: **Truncate legends** when they are too long:
- Number of lines in the `tap` field is truncated to at most 3; if >3, 3rd line will be replaced with `…`
- Length of each legend (in a single line) is truncated to `1.7 × draw_cfg.shrink_wide_legends` (by default `11`) chars. If longer than that, the last visible char will be replaced with `…`, similar to above

Fixes

- Fix parsing of devicetree arrays when they are split to multiple lines and multiple `<...>` blocks

Misc

- Add Ferris physical layout with rotated thumb keys, applying to all Ferris/Sweep variants
- This will be used by default; if you don't want this you can use [the original `info.json`](https://github.com/qmk/qmk_firmware/blob/master/keyboards/ferris/sweep/info.json) with `keymap draw --qmk-info-json` argument, or the new "Layout Override" dialog in the web app
- Add Swoop physical layout
- Add support for generic physical layouts via `layout: {qmk_keyboard: generic/<layout_name>}`, which are specially fetched from https://github.com/qmk/qmk_firmware/tree/master/layouts/default/ (where `layout_name` is a subfolder name)

**Full Changelog**: https://github.com/caksoylar/keymap-drawer/compare/v0.16.0...v0.17.0

This release contains two major new features: parsing of modifier functions and dark mode support! It also contains a couple of smaller breaking changes, so make sure to check them out below.

Features

Parsing
- Added support for **parsing modifier functions** like `LC(V)` in ZMK and `LCTL(KC_V)` in QMK, used for sending modified keycodes like <kbd>ctrl</kbd>+<kbd>v</kbd> events with one key press
- By default these will be converted to `Ctl+ V` representation, this is configurable via the new [`parse_config.modifier_fn_map` setting](https://github.com/caksoylar/keymap-drawer/blob/v0.16.0/CONFIGURATION.md#modifier_fn_map)
- You can also assign special names to specific mod combinations, like `Hyper+ X` for `LC(LS(LG(LA(X))))`
- **Breaking change:** If you were using entries like `LC(V): Paste` in `parse_config.zmk_keycode_map` or `C(V): Paste` in `qmk_keycode_map`, they will no longer work because `LC` and `V` parts are processed separately. If you want to preserve the mapping, instead define them in `raw_binding_map` like `&kp LC(V): Paste` for ZMK or `C(KC_V): Paste` for QMK.
- Added parsing for QMK tap-toggle layers `TT(LAYER)` by m-demare in https://github.com/caksoylar/keymap-drawer/pull/87

Drawing
- Added **dark mode support**, by snoyer!
- This is implemented as a special set of [CSS overrides](https://github.com/caksoylar/keymap-drawer/blob/v0.16.0/CONFIGURATION.md#svg_style_dark1) that get added to the final CSS if the new `draw_config.dark_mode` setting is on, or added under `media (prefers-color-scheme: dark)` when set to `"auto"` to fit the web page or OS light/dark setting.
- The default is `false` which means light mode only, so this is not a breaking change for CLI users. The [web app](https://caksoylar.github.io/keymap-drawer/) overrides it to `"auto"`.

| `dark_mode: false` | `dark_mode: true` | `dark_mode: auto` |
| - | - | - |
|  |  |  |

- Added support for **appending footer text** to produced SVGs using [`draw_config.footer_text`](https://github.com/caksoylar/keymap-drawer/blob/v0.16.0/CONFIGURATION.md#footer_text)
- The value goes into a `<text>` element but it isn't escaped, so you can add e.g. links using `<a>` tags
- The default value is empty so this isn't a breaking change for CLI users. The [web app](https://caksoylar.github.io/keymap-drawer/) overrides it to "<text>Created with <a href="https://github.com/caksoylar/keymap-drawer">keymap-drawer</a></text>"

- Added support for **using [Phosphor Icons](https://phosphoricons.com)**, check out how to use them in [the README section](https://github.com/caksoylar/keymap-drawer/blob/v0.16.0/README.md#custom-svg-glyphs)

CLI

- Added `-o`/`--output` parameter to all `keymap` subcommands, to write their output to the given path rather than stdout
- Using these parameters are recommended if you are running the CLI on Windows, since they ensure the outputs are written with `utf-8` encoding rather than the locale-specific one (which can break many things). This release also incorporates a couple fixes that ensure operation in Windows (tested in powershell)
- **Breaking change:** `keymap draw --ortho-layout` no longer supports the `-o` short form since that is taken over by the output parameter

**Full Changelog**: https://github.com/caksoylar/keymap-drawer/compare/v0.15.0...v0.16.0

This is a smaller release with a couple of features that I wanted to release quickly: Automatic QMK keyboard mappings and changes to the config dumping.

Features

- **Breaking change**: Added a [new mapping layer](https://github.com/caksoylar/keymap-drawer/commit/e20025afef2230dca5aad6ac098337247b73e1cf#diff-06c2dc3fe896463bff49cceaf51980c3c9133d9384f855beb2ef3477da23e372) for QMK keyboard names
- This allows us to map given QMK keyboard name prefixes to other keyboard names during drawing
- It is currently used to map Corne, Lily58, Sofle and Kyria variants to the versions with rotated keys in [`qmk_layouts` folder](https://github.com/caksoylar/keymap-drawer/tree/main/resources/qmk_layouts)
- This is a breaking change if you were using e.g. `qmk_keyboard: crkbd/rev1` rather than `qmk_keyboard: corne_rotated` -- there is no way to prevent this mapping right now, so let me know if you really want the old behavior instead
- **Breaking change**: Config dumps via `keymap dump-config` no longer include certain "internal" draw config options, namely `svg_style`, `glyph_urls` and `use_local_cache`
- The rationale is that these shouldn't typically touched by users, and having them be present in all configs creates issues when default values change (such as when 78 is fixed)
- You can still override them in your config by manually including the keys, the functionality didn't change

Fixes

- Specially parse `0xff` as a special value in the `layers` property of the combo spec in ZMK keymaps, to mean all layers. This is used by [nodefree-config](https://github.com/urob/zmk-nodefree-config/blob/973f717/helper.h#L65)

Misc

- Add Kyria physical layout with rotated keys, by micampe
- Log what config args are used during ZMK drawing workflow

**Full Changelog**: https://github.com/caksoylar/keymap-drawer/compare/v0.14.1...v0.15.0