Zrb

- **Feature: new built-in developer-utility task groups** (`builtin/hash.py`, `builtin/datetime.py`, `builtin/url.py`, `builtin/json.py`, `builtin/case.py`, `builtin/cron.py`, `builtin/hex.py`, `builtin/number.py`, registered in `builtin/group.py` and `builtin/__init__.py`):
- `hash` — `hash` (text), `sum` (file, streamed in 8 KiB chunks), and `hmac`, each over `sha256`/`sha1`/`sha224`/`sha384`/`sha512`/`md5` via `hashlib.new`. Generalizes the existing md5-only group.
- `time` — `now`, `to-iso` (epoch→ISO 8601), and `to-epoch` (ISO 8601→epoch); naive ISO datetimes are interpreted as UTC.
- `url` — `encode`/`decode` (percent-encoding) and `parse` (URL split into a JSON object of scheme/host/port/path/query/fragment).
- `json` — `format`, `minify`, `validate`, `get` (dotted-path extraction, e.g. `user.roles[0]`), plus `to-yaml`/`from-yaml`.
- `case` — `convert` (snake/camel/pascal/kebab/constant/title) and `slugify` (accent-stripping, URL-friendly).
- `cron` — `parse` validates an expression and lists upcoming run times, reusing `zrb.util.cron.match_cron` (no new dependency).
- `hex` — `encode`/`decode` (ASCII↔hex, tolerating spaces and a `0x` prefix) and `dump` (offset + hex + ASCII hexdump).
- `number` — `convert` between bases 2/8/10/16.
- All new tasks are stdlib/`pyyaml`-only, so no new runtime dependency was added.

- **Feature: secure random generators** (`builtin/random.py`): added `random password`, `random token` (`secrets.token_urlsafe`), and `random string`, all backed by the `secrets` module.

- **Feature: `core-coding` skill gains a runtime-debugging / observability companion** (`llm_plugin/skills/core-coding/`):
- New `workflows/observability.md` companion for diagnosing a *running or deployed* system that can't be reproduced locally — distinct from the local-failure `workflows/debug.md` (which now cross-links it). Covers core dumps (`gdb`/`coredumpctl`, Go/Python specifics), memory & heap dumps (Java `jcmd`/MAT, Python `py-spy`/`tracemalloc`, Node heap snapshots, Go `pprof`), `kubectl` triage (exit-code reading — 137/OOMKilled, 139/SIGSEGV, CrashLoopBackOff — `--previous` logs, `kubectl debug`, `kubectl cp` to pull dumps), Grafana/Prometheus (RED/USE, PromQL for p99 / working-set / restarts), and Elasticsearch/Kibana (KQL, trace-id correlation, the `_search` API), chained metrics → logs → runtime → dump.
- Two portable, read-only Python helper tools (stdlib only): `tools/k8s-triage.py` (one-shot pod/deployment triage — status, events, `--previous` + current logs, `top`, exit-reason guidance; accepts a pod, `deploy/<name>`, or `label=value` selector) and `tools/coredump-bt.py` (all-thread `gdb` backtrace from a `<binary> <core>` pair or `--systemd <exe|pid>` via `coredumpctl`). Registered as a trigger row in `core-coding/SKILL.md` and auto-discovered as companion files.

- **Improvement: JWT decode without a secret** (`builtin/jwt.py`): `jwt decode` now defaults to inspect-without-verify (the jwt.io workflow) — paste a token and read its claims with no secret. Pass `--verify` to check the signature. Well-known timestamp claims (`exp`/`iat`/`nbf`/`auth_time`) are printed in human-readable UTC alongside their raw values.

- **Improvement: `http request` returns a pipe-friendly body** (`builtin/http.py`): the task now returns `response.text` (was a `requests.Response`, which printed as `<Response [200]>` on stdout), so `zrb http request ... | jq` works. Decorations stay on stderr. Added `body-format` (`json`/`form`/`raw`), `params` (query string as JSON), and `timeout` inputs; `HEAD`/`OPTIONS` added to the method list.

- **Improvement: Base64 URL-safe alphabet and correct validation** (`builtin/base64.py`): `encode`/`decode`/`validate` accept `--url-safe` (`-_` alphabet). `validate` now checks true base64 well-formedness with `validate=True` instead of requiring the payload to be UTF-8 text, so base64 of binary data (images, gzipped blobs) validates correctly.

- **Improvement: Claude-Code command hooks (e.g. peon-ping) now work end-to-end** (`llm/hook/hook_creators.py`, `llm/hook/hook_loader.py`, `llm/hook/manager/loader_mixin.py`, `llm/agent/run/runner.py`):
- **Event payload on stdin (ADR-0066).** `create_command_hook` now writes the Claude-shaped event JSON (`HookContext.to_claude_json()` — `hook_event_name`, `session_id`, `cwd`, `tool_name`, …) to the subprocess' stdin via `communicate(input=…)`. Stdin-driven hooks like peon-ping read `json.load(sys.stdin)["hook_event_name"]` and ignore env vars, so without this they never fired. The existing `CLAUDE_*` env vars are unchanged.
- **`settings.json` is now a hook source.** `_collect_hook_paths` also reads `.claude/settings.json` and `.claude/settings.local.json` (project and home), where Claude Code — and drop-in tools like peon-ping — register their hooks. The nested `hooks` block is parsed by the existing `_parse_claude_format`; other settings keys are ignored.
- **`Stop` fires on natural turn completion.** `run_agent` fires `HookEvent.STOP` when a turn finishes and control returns to the user, not only on manual interrupt from the TUI. This is the per-turn "done" signal completion sounds/notifications listen on. The deferred-wait path does not fire it, and manual interrupts still fire `Stop` from the TUI (no double-fire).
- Unknown events in a Claude config (`SubagentStart`/`SubagentStop`, which zrb does not yet emit) are now skipped at `debug` level instead of warning on every load.
- **`PermissionRequest` event + attention notifications** (`llm/hook/types.py`, `llm/agent/run/deferred_calls.py`, `llm/tool/ask.py`): added `HookEvent.PERMISSION_REQUEST`, fired from the approval cascade (`_resolve_approval`) exactly when a tool call reaches an interactive prompt — after every auto-resolve path (always-approve, tool/permission policy, YOLO), so it never fires for auto-approved calls. `AskUserQuestion` (auto-approved, so it skips that path) fires a `Notification` with `notification_type='elicitation_dialog'` from `ask_user_question`. Both ring peon-ping's `input.required` sound, so tool-approval and question prompts now notify you.
- **Fix: command-hook working directory.** `create_command_hook` now `expanduser`s the cwd and falls back to the inherited cwd when the directory is missing, instead of passing it straight to `create_subprocess_shell`. The UI's `Notification` hook also no longer passes the *display* cwd (`~/zrb`, home collapsed to `~`) — which the OS cannot resolve — so hooks stopped failing with `[Errno 2] No such file or directory: '~/…'` on every streamed output chunk.
- **Fix: async command hooks are truly fire-and-forget** (`llm/hook/manager/manager.py`): `execute_hooks` now dispatches `async` command hooks as detached tasks on the running event loop instead of inside the thread executor's short-lived `asyncio.run` loop (which cancelled them immediately) and then *awaiting* the subprocess. Previously every `async` peon hook — including the `Notification` hook that fires per streamed output chunk — blocked the executor up to its timeout, producing a storm of `Hook execution timed out after 10s` warnings and multi-second stalls per turn. They now return instantly and complete in the background.
- **Fix: command-hook timeout is enforced.** `create_command_hook` wraps `communicate()` in `wait_for` and `kill()`s the subprocess on timeout. The thread-pool executor's own `wait_for` cannot interrupt a blocked worker thread, so a hook (or a child it forks, e.g. peon-ping's audio player) that hung used to block well past its configured timeout.
- **Fix: hooks no longer register (and fire) twice** (`llm/hook/hook_loader.py`): `get_search_directories` now dedups by resolved path. `$HOME` is searched by both the home tier and the project upward-walk whenever cwd is under `$HOME`, so `~/.claude/*` hooks were discovered and registered twice — every event fired twice (e.g. two peon-ping toasts per prompt).
- **Fix: hook floods no longer exhaust file descriptors or storm timeouts** (`llm/hook/manager/manager.py`, `llm/hook/hook_creators.py`, `llm/ui/default/output_mixin.py`): three reinforcing fixes after fire-and-forget hooks exposed a flood. (1) Fire-and-forget command hooks are now bounded — a semaphore caps concurrent subprocesses and a backlog ceiling sheds excess — so a burst can't spawn hundreds of `peon.sh` at once (`[Errno 24] Too many open files`) or back up a serialized tool into a `timed out after 10s` storm. (2) Injected `CLAUDE_*` env values are size-capped, so a large `event_data` (e.g. full message history on `SessionStart`/`Stop`/`SessionEnd`) no longer overflows the OS exec limit (`[Errno 7] Argument list too long`); the full payload still arrives on stdin. (3) The UI no longer fires a `Notification` hook per streamed output chunk — that conflated "output produced" with the Claude `Notification` event (attention/permission), spawning a subprocess per chunk for no benefit (peon-ping suppresses it as unrecognized). Genuine attention notifications fire at the right moments (`PermissionRequest`, `elicitation_dialog`).
- **Fix: `ZRB_HOOKS_ENABLED` is honored as a global kill-switch** (`llm/hook/manager/manager.py`): the documented flag was defined but never read, so toggling it did nothing. `execute_hooks` now returns immediately — no hook fires and the filesystem is not scanned — when `CFG.HOOKS_ENABLED` is off (default stays `on`). `docs/configuration/llm-config.md` corrected (default `on`, not `1`).

- **Tests**: new coverage for every new built-in (`test_base64.py`, `test_case.py`, `test_cron.py`, `test_datetime.py`, `test_hash.py`, `test_hex.py`, `test_json.py`, `test_number.py`, `test_url.py`, plus `test_jwt.py` / `test_random.py` / `test_http.py` extensions) and for the hook changes (`test_functionality.py`: stdin payload, `settings.json` loading, async fire-and-forget non-blocking, timeout-kill, oversized-env drop, search-dir dedup; `test_deferred_calls.py` / `test_runner.py`: `PermissionRequest` / `Stop` / `Notification` firing).
- **Hook test isolation**: a session-scoped autouse fixture in `test/conftest.py` (plus `test/llm/agent/run/conftest.py`) pins the module-level `hook_manager` singleton to no search dirs and reloads it, so the suite never discovers and spawns the developer's real `~/.claude` hooks (e.g. peon-ping) — which previously made runs crawl and hang asyncio's subprocess teardown. `test/llm/hook/test_executor.py`'s timeout test was shortened (~2s → ~0.3s) since the worker thread is non-cancellable and `shutdown(wait=True)` must join it.
- **Build/test config**: `pyproject.toml` `norecursedirs` now excludes `examples/` (its scripts are demos, not part of the suite) and added the `httpx2` dev dependency; removed the redundant `examples/llm-hooks/test_hooks.py` (a never-collected demo whose scenarios are already covered by `test/llm/hook/`); `.coveragerc` omits skill tool scripts (`**/llm_plugin/skills/**/tools/*.py`), which are shipped agent-runtime content rather than unit-tested library code.

- **Build: migrate `pyproject.toml` to the PEP 621 `[project]` table** (`pyproject.toml`, `scripts/build_pypi_readme.py`, `zrb_init.py`):
- Moved package metadata, runtime `dependencies`, `optional-dependencies` (extras), `scripts`, and `urls` out of `[tool.poetry.*]` into the standard `[project]` table. Only Poetry-specific keys (`exclude`, the `dev` dependency group, `build-system`) remain under `[tool.poetry]`. The two in-repo version readers were repointed from `["tool"]["poetry"]` to `["project"]`: `zrb_init.py` (`_VERSION`) and `scripts/build_pypi_readme.py` (which now also reads `["project"]["urls"]["repository"]`).
- **Removed the aggregate `all` extra.** Under legacy `[tool.poetry.extras]` it referenced *extra names* (`bedrock`, `huggingface`, `xai`, `voyageai`) that Poetry silently ignored, so `pip install zrb[all]` never pulled boto3/botocore (the AWS SDK) and other extra-only packages; rewriting it as explicit requirements would just duplicate — and risk drifting from — every version pin (it had already lost the `langchain-core>=1.3.3` security floor). Since `poetry install --all-extras` installs every extra without a meta-extra and `install.sh` installs zrb with no extras, `all` was dropped entirely. pip users enumerate the extras they want (`pip install "zrb[rag,...,voyageai]"`).
- Relocated the centralized security-pin comment block onto the transitive dependency that carries each pin, inline within its owning extra (`pyasn1` in `vertexai`; `aiohttp` in `xai`/`voyageai`; `langchain-core`/`langchain-text-splitters`/`langsmith` in `voyageai`).

- **Improvement: gate `poetry lock` on consistency in `project.sh`**:
- `project.sh` ran a bare `poetry lock` on every shell `source`, which re-resolves the whole dependency graph from PyPI even when `poetry.lock` is already consistent — a multi-minute cost, made heavier by the now-complete `all` extra. The lock step now runs only when the fast hash-compare reports drift (`poetry check --lock >/dev/null 2>&1 || poetry lock`), so an unchanged setup skips the re-resolve entirely. The companion `~/borg/init-borg.sh` install step was given the same guard.

- **Performance: prompt caching restored via stable system prompt + per-turn `<live-context>`** (`prompt/system_context.py`, `prompt/manager.py`, `task/llm_task.py`, `agent/run/runner.py`, `agent/subagent/manager/manager.py`; ADR-0065):
- The system prompt is sent ahead of the conversation history, so its bytes must be identical across turns for any provider's prefix cache to hit. The `system_context` section opened with a second-resolution `- Time:` line plus git status / pending todos / active worktree, all of which change every turn — diverging the prefix and forcing a full cache miss on every request (observed `prompt_cache_hit_tokens: 0`), including the growing conversation history.
- Split `system_context` by lifecycle: it now renders only session-invariant facts (OS, CWD, project markers, tools, model identity) into the cached system prompt, plus a stable anchor explaining the `<live-context>` contract. The volatile per-turn state (time, git, todos, worktree, mode, interactivity) and the per-turn ambient-state wiring (session / interactive / stale-worktree) moved to a new `render_live_context()`.
- `PromptManager.create_live_context()` wraps that body as `<live-context>…</live-context>`; `LLMTask.get_live_context()` renders it and `run_agent(live_context=…)` appends it to the end of the current user turn (`_append_live_context`, handling text / multimodal / empty turns). The block is append-only and frozen into history, so the system-prompt-plus-history prefix stays byte-stable and caches across turns.
- Sub-agents are single-turn, so the block is folded back into their inherited system prompt (`_build_inherited_prompt`) when they inherit `system_context` — preserving prior behavior with no caching downside.

- **Fix: history manager memory leak** (`file_history_manager.py`): The in-RAM conversation cache grew unboundedly across a session. Added an LRU eviction cap (`_MAX_CACHED_CONVERSATIONS = 8`) with dirty-entry tracking so unsaved updates are never dropped. The `_dirty` set is cleared on `save()`; clean entries reload losslessly from disk on next access.

- **Fix: LSP project root cache unbounded growth** (`lifecycle_mixin.py`): The per-file project-root directory walk cache had no size bound. Added `_MAX_PROJECT_ROOT_CACHE = 4096` with oldest-entry eviction. Precision doesn't matter at this level — unboundedness does.

- **Fix: denial reason accepts arbitrarily large payloads** (`handler.py`, `response_handler/default.py`, `truncate.py`): A mis-submitted input (whole screen buffer, pasted document) entered the conversation history as a tool result. Added `truncate_chars(text, 500)` with a `...[TRUNCATED N chars]` marker in `src/zrb/util/truncate.py`, applied in both the inline approval handler and the default response handler.

- **Fix: Enter key with focus on output pane submits pane content** (`keybindings_mixin.py`): Tab/F6 puts focus on the read-only output buffer. Enter there would resolve a pending confirmation or submit the entire pane content (banner, help, transcript) as user input. Enter now unconditionally refocuses the input field when focus is elsewhere.

- **Feature: logged warning for empty custom prompt sections** (`manager.py`): When a name in `include_sections` / `ZRB_LLM_INCLUDE_SECTIONS` resolves as neither built-in, registered provider, nor existing markdown file, the section is now composed as empty (instead of the previous silent no-op) and a warning is logged or printed at compose time — so a misspelled section name is diagnosable.

- **Improvement: prompt system weight reduction** (30 files across `prompt/`, `tool/`, `tool_call/`, `common_tools.py`, `llm_plugin/agents/`, `llm_plugin/skills/`, `AGENTS.md`):
- Stripped redundant usage instructions from tool docstrings and tool guidance that duplicate the Tool Usage Guide.
- Shifted Skill Activation policy and authority rules from the skills catalogue into the centralized Operating Rules section (single source of truth).
- Standardized activation language across all sub-agent definitions (code-reviewer, generalist, researcher) to reference the Operating Rules table rather than restating per-agent rules.
- Removed duplicative `when_to_use`/`key_rule` entries where the tool name is self-describing; tightened remaining guidance to cross-tool choice logic only.
- Replaced "activate before X" phrasing with "activate when the deliverable is X" across all skill descriptions, matching the Skill Activation table semantics.
- Stripped "Deliver complete outputs" from persona (implied by the quality rules), "Wait for agreement" from persona (covered by the Working Loop Frame step), and other weight lines.
- Removed obsolete `__doc__` reassignment in `delegate.py` (the docstring is already on the function) and outdated `Mandates` section in `code.py` (redundant with tool guidance).
- Tightened delegation guidance: removed motivational framing around fidelity/cost; kept only the scope-clamp rule.
- Clarified that `core-coding` companions are defaults, not absolutes — explicit project guidelines override them.
- Updated `git-summary/SKILL.md` to: default to drafting only, commit/PR only on explicit request.
- Reduced total prompt weight by ~143 lines (net diff: +253 -143).

- **Feature: arrow-key selection UI for AskUserQuestion**:
- The default full-screen chat UI and `StdUI` now render `AskUserQuestion` as an interactive, arrow-key-selectable list (↑/↓ to move, Enter to confirm) instead of requiring the user to type an option number. Multi-select questions use Space to toggle; a synthetic "✎ Type my own answer…" row drops to free-text, and in multi-select the typed text is appended to the already-checked options.
- New optional `UIProtocol.ask_user_choice(spec: ChoiceSpec)` method (`tool_call/ui_protocol.py`). `BaseUI.ask_user_choice` provides a default that formats the spec as numbered text and delegates to `ask_user`, so the web/`SimpleUI`/`MultiUI`/sub-agent paths keep the existing type-a-number behavior unchanged.
- Default UI: new `SelectionMixin` (`ui/default/selection_mixin.py`) renders the widget as an in-layout `Float` (no nested prompt-toolkit `Application`); `ConfirmationMixin`'s queue was generalized to `(future, prompt, spec)` so text confirmations and choices share one serialization path. `tool/ask.py` builds a `ChoiceSpec` per question and routes through `ask_user_choice` (falling back to `ask_user` for UIs that predate it).
- Presentation: the streamed `🧰` tool-call line suppresses `AskUserQuestion`'s (large) args payload (`util/stream_response.py`) since the widget renders it; the float is full-width with an opaque panel background and a highlight bar on the cursor row (`app/style.py`) so the streaming output behind it no longer bleeds through; the question is echoed into scrollback with its answer only on resolve (no duplicate while the widget is open).
- Tests: `test/llm/ui/default/test_selection_mixin.py`, plus extended `test/llm/ui/test_std_ui.py`, `test/llm/tool/test_ask.py`, and `test/llm/util/test_stream_response.py`.

- **Feature: opt-in filesystem sandbox for LLM tool calls (ADR-0063)**:
- New `zrb/llm/sandbox/` package: one `SandboxPolicy` drives two enforcement layers — a Python-level FS gate (`_sandbox_gate` in `agent/common.py`, right after `_permission_gate`) that blocks writes outside the writable roots (`EDIT`/`UNKNOWN` tools) and reads of credential directories (all tools), and an OS-level wrapper for `Shell`/`Bash`/`ShellBackground` (`sandbox-exec` + generated SBPL on macOS, `bwrap` on Linux). Network stays open in v1; off by default (`ZRB_LLM_SANDBOX_ENABLED=false`).
- Config: `ZRB_LLM_SANDBOX_ENABLED` / `OS_SHELL` / `WRITABLE_PATHS` / `DENY_READ_PATHS` / `FALLBACK` / `ALLOW_ESCAPE` (new `LLMSandboxMixin`). Where no OS mechanism exists (Windows, Linux without bwrap), `FALLBACK=warn` runs unsandboxed with a visible warning, `deny` refuses — never silent.
- Escape hatch: `dangerously_skip_sandbox` on the shell tools — never auto-approved (`bash_validation`/`auto_approve` always route it to a human), blockable via `ALLOW_ESCAPE=false`.
- Plumbing mirrors permissions: `LLMTask(sandbox=...)`, `run_agent(sandbox_policy=...)`, `current_sandbox_policy` ContextVar (sub-agent inheritance).
- Shell PID-tracking wrapper now falls back to `$$` when `ps` is unavailable (macOS Seatbelt cannot exec setuid binaries) and records the shell's own PID for exclusion under wrappers.
- Docs: `docs/advanced-topics/sandbox.md`, sandbox section in `docs/configuration/llm-config.md`, ADR-0063. Tests: `test/llm/sandbox/` incl. platform-conditional integration tests (real Seatbelt/bwrap runs).

- **Fix: AskUserQuestion prompt never rendered (stuck at "waiting for confirmation")** (`ui/default/confirmation_mixin.py`): `ask_user` set `_current_confirmation` *before* appending the prompt, so the prompt hit `OutputMixin.append_to_output`'s buffer guard (pending-confirmation + thinking → buffer to avoid interleaving main-agent tokens) and was buffered away instead of shown. The user saw the `🧰` tool-call line and "waiting for confirmation" but no question. The prompt is now appended *before* marking the confirmation pending, in both `ask_user` and `_activate_next_confirmation`.

- **Fix: AskUserQuestion gated behind a redundant approval prompt** (`tool/ask.py`, `tool_call/always_approve.py`, `agent/run/deferred_calls.py`, ADR-0062): `AskUserQuestion` *is* the user interaction, but as a deferred tool it went through the approval cascade — asking "Allow tool execution?" before the question itself rendered. Auto-approval was only wired via a single `auto_approve("AskUserQuestion")` entry in `builtin/llm/chat.py`, so delegated sub-agents, the web/API runner, and bare `LLMTask`s still prompted (or left the question un-surfaced). Auto-approval is now **intrinsic to the tool**: it self-registers via `register_always_auto_approve("AskUserQuestion")`, and `_resolve_approval` honors that registry as Priority 0 in every path. The redundant `chat.py` entry was removed (single source of truth).