Ragmill

Fixed
- Documentation now points at `ragmill setup-chat` for the local model everywhere, rather than `pip install "ragmill[chat]"` — a source build, and the command that fails on Windows. It was still the headline route in the README install list, both backend tables, and the installation-guide extras table.
- The quickstarts in `docs/index.md`, `docs/quickstart.md` and the README CLI list went straight from `pip install` to `ragmill chat`, walking a new reader into the missing-model error. They now include the one-time `ragmill setup-chat` step.
- `ragmill setup-chat` was missing from the CLI reference command table.
- Added tests asserting that no doc prints the wheel-index command without `--only-binary`, and that any quickstart showing `ragmill chat` also mentions `setup-chat` — the previous fix corrected three files by hand and missed six other places.

Added
- `ragmill setup-chat` installs the local chat model runtime for you. It prints the package, the third-party index it comes from, and the exact pip command, then asks before installing anything (`--yes` skips the prompt, and is required when stdin is not a terminal). It installs into the interpreter running RAGMill rather than whichever `pip` is first on `PATH`, and verifies the module is importable afterwards instead of trusting pip's exit code.

This cannot happen during `pip install`: a wheel is unpacked, never executed, so Python packaging has no post-install hook — deliberately, since npm's `postinstall` became a supply-chain attack vector. Making it an explicit command also keeps the non-PyPI index visible and consented to, the way `playwright install` and `python -m spacy download` do.

Fixed
- **The documented command for installing the local model did not work.** `pip install llama-cpp-python --extra-index-url …` resolves to `llama_cpp_python-0.3.34.tar.gz`, not a wheel: `--extra-index-url` merges both indexes and pip picks the highest version across them, and PyPI carries a newer sdist-only release than the wheel index carries wheels. So the workaround for the Windows build failure triggered that very build failure. Every place this command appears — the CLI error, README, installation guide, chat guide — now passes `--only-binary llama-cpp-python`, which makes pip skip versions that have no wheel.

Changed
- `ragmill chat` now checks the configured backend **before** starting the REPL. Previously a missing local model was only discovered after you had typed a question — the question was lost and a raw Python traceback was printed for what is simply an optional package not being installed yet. It now exits cleanly with status 1 and a message.
- The "local model not installed" message leads with the single command that installs it (`pip install llama-cpp-python --extra-index-url …`) and no longer suggests the Gemini or OpenAI backends. Someone running the local backend asked for local chat; offering a hosted service that needs an API key is a detour, not an answer. It also no longer suggests `pip install "ragmill[chat]"`, which is a source build and the exact thing that fails on Windows.

Fixed
- Text files are now decoded by their actual encoding instead of being read as UTF-8 with `errors="ignore"`. The old behaviour corrupted files silently, which mattered most on Windows because it is what Notepad writes:
- **"Unicode" (UTF-16)** decoded as UTF-8 produced NUL-interleaved mojibake (`T\x00h\x00e\x00…`). The file ingested "successfully", but its chunks embedded as noise and never matched a query — a `.txt` file that appeared not to work at all.
- **"ANSI" (cp1252)** lost every non-ASCII byte, turning `costs £50` into `costs 50` — a silent change of meaning rather than a visible failure.
- **UTF-8 with BOM** left a stray `` at the head of the first chunk.

Byte-order marks are now honoured (UTF-8/16/32, and the BOM is stripped rather than left in the text), BOM-less UTF-16 is detected via interior NULs, and non-UTF-8 files fall back to cp1252 then latin-1 **with a warning** naming the encoding used. This applies to every text reader — plain text (`.txt`, `.md`, `.log`, `.rst`), `.csv`/`.tsv`, `.html`, and `.rtf` — all four of which had the same flaw.

Fixed
- Errors for missing system binaries now name a command for the user's own OS. `tesseract` and poppler's `pdftoppm` cannot come from pip, and the messages previously suggested `brew install` on every platform — useless on Windows and Linux. The scanned-PDF error also mentions `enable_ocr=False` for opting out rather than only how to opt in.
- The "local chat backend unavailable" error no longer points solely at `pip install "ragmill[chat]"`, which is the command that fails on Windows. It now leads with the prebuilt-wheel index, explains why the package is not in `[all]`, and offers the hosted Gemini/OpenAI backends that need no local model.
- `pip install ragmill[all]` no longer fails on a clean machine. The `all` (and `dev`) extras pulled `llama-cpp-python`, which ships no PyPI wheels for recent versions, so pip fell back to a 70MB+ sdist that vendors llama.cpp — that needs a C++ toolchain, and on Windows the vendored tree exceeds the 260-char `MAX_PATH` limit, aborting the whole install with `OSError: [Errno 2] No such file or directory`. The local LLM is now opt-in: install a prebuilt wheel with `pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu` (no compiler needed), or use the hosted `chat-gemini` / `chat-openai` backends, which are included in `[all]`. `pip install "ragmill[chat]"` still works if you have CMake and a C++ toolchain — and, on Windows, long paths enabled.
- Source distributions are built from an explicit allowlist. The sdist previously included every file `.gitignore` did not exclude, so a locally built tarball could sweep in a maintainer's untracked working directories. Releases built by CI were unaffected.

Added
- Extended file-format support: `.csv`/`.tsv` (stdlib, no extra), plus `.html`/`.htm`, `.rtf`, `.xlsx`, and `.pptx` via the new `office` extra (`pip install ragmill[office]`).
- OCR support via the new `ocr` extra (`pip install ragmill[ocr]`): text extraction from images (`.png`, `.jpg`, `.jpeg`, `.tiff`, `.bmp`, `.gif`) and automatic fallback to OCR for scanned/image-only PDFs (requires the system `tesseract` binary, and `pdftoppm`/poppler for PDFs).
- DOCX extraction now also captures table cell text, not just paragraphs.

Changed
- Chat answers now lead with a direct answer followed by a brief 2–4 sentence explanation, and no longer embed bracketed citation markers (`[1]`, `[report.pdf]`) — sources are listed separately by the caller.
- The ingestion engine now skips files that yield no extractable text (e.g. scanned PDFs with no OCR result) with a warning instead of storing an empty document.