Shot-scraper

- `shot-scraper javascript` can now optionally [load scripts hosted on GitHub](https://shot-scraper.datasette.io/en/stable/javascript.html#running-javascript-from-github) via the new `gh:` prefix to the `shot-scraper javascript -i/--input` option. 173

Scripts can be referenced as `gh:username/repo/path/to/script.js` or, if the GitHub user has created a dedicated `shot-scraper-scripts` repository and placed scripts in the root of it, using `gh:username/name-of-script`.

For example, to run this [readability.js](https://github.com/simonw/shot-scraper-scripts/blob/main/readability.js) script against any web page you can use the following:

bash
shot-scraper javascript -i gh:simonw/readability \
https://simonwillison.net/2025/Mar/24/qwen25-vl-32b/

- New options for the `shot-scraper har` command: `--wait`, `--wait-for` and `--javascript` - similar to those same options on `shot-scraper shot`. 171
- Documentation now recommends using `shot-scraper multi --har-zip` for larger collections of files to avoid a Playwright crash if the `.har` JSON grows too large. 170

- New `shot-scraper har URL` command for [creating an HTML Archive](https://shot-scraper.datasette.io/en/stable/har.html) of a page. #146
- `shot-scraper multi --har` and `--har-zip` and `--har-file` options for [creating an HTML Archive of multiple pages](https://shot-scraper.datasette.io/en/stable/multi.html#multi-har). 166
- `shot-scraper pdf` now has `--wait-for` and `--timeout` options. Thanks, [miccou](https://github.com/miccou). #161

- Several new features for the YAML configuration used by [shot-scraper multi](https://shot-scraper.datasette.io/en/stable/multi.html):
- You can now add a `- server: python -m http.server 8003` block to [start a server running](https://shot-scraper.datasette.io/en/stable/multi.html#running-a-server-for-the-duration-of-the-session) before screenshots are taken. The PID for this server will be recorded and the server automatically terminated when the command completes, unless you specify the `--leave-server` option in which case it will be left running, useful for debugging. 156
- The `sh: shell command` or `python: python code` blocks can [specify Python or shell commands](https://shot-scraper.datasette.io/en/stable/multi.html#running-custom-code-between-steps) to run before a screenshot is taken. This means a YAML script can make modifications to the environment in between screenshots, useful for things like progressive tutorials. 155
- Fixed a bug that occurred if a `max-width` was accidentally applied to the `<div>` used for region screenshots. Thanks, [Johann Klähn](https://github.com/kljohann). #143
- Documented that `shot-scraper` will quit with an error if a `--wait-for` expression has not resolved in 30s.

- New `--auth-username x --auth-password y` options for each `shot-scraper` command, allowing a username and password to be set for HTTP Basic authentication. [140](https://github.com/simonw/shot-scraper/issues/140)
- `shot-scraper URL --interactive` mode now respects the `-w` and `-h` arguments setting the size of the browser viewport. Thanks, [mhalle](https://github.com/mhalle). [#128](https://github.com/simonw/shot-scraper/issues/128)
- New `--scale-factor` option for setting scale factors other than 2 (for retina). Thanks, [Niel Thiart](https://github.com/nielthiart). [#136](https://github.com/simonw/shot-scraper/issues/136)
- New `--browser-arg` option for passing extra browser arguments (such as `--browser-args "--font-render-hinting=none"`) through to the underlying browser. Thanks, [Niel Thiart](https://github.com/nielthiart). [#137](https://github.com/simonw/shot-scraper/issues/137)

- New `--bypass-csp` option for bypassing any Content Security Policy on the page that prevents executing further JavaScript. Thanks, [Brenton Cleeland](https://github.com/sesh). [#116](https://github.com/simonw/shot-scraper/pull/116)
- Screenshots taken using `shot-scraper --interactive $URL` - which allows you to interact with the page in a browser window and then hit `<enter>` to take the screenshot - it no longer reloads the page before taking the shot (which ignored your activity). [125](https://github.com/simonw/shot-scraper/issues/125)
- Improved accessibility of [documentation](https://shot-scraper.datasette.io). Thanks, [Paolo Melchiorre](https://github.com/pauloxnet). [#120](https://github.com/simonw/shot-scraper/pull/120)