Shot-scraper

- New `shot-scraper accessibility --timeout` option, thanks [Ben Welsh](https://github.com/palewire). [#59](https://github.com/simonw/shot-scraper/pull/59)
- `shot-scraper auth --browser` option for authentication using a browser other than Chromium. [61](https://github.com/simonw/shot-scraper/issues/61)
- Using `--quality` now results in a JPEG file with the correct `.jpg` extension. Thanks, [Ian Wootten](https://github.com/iwootten). [#58](https://github.com/simonw/shot-scraper/pull/58)
- New `--reduced-motion` flag for emulating the "prefers-reduced-motion" media feature. Thanks, [Ryan Murphy](https://github.com/rdmurphy). [#49](https://github.com/simonw/shot-scraper/issues/49)

- New `-b/--browser` option for the `shot-scraper install`, `shot`, `multi` and `javascript` commands. This can be used to install and run alternative browsers `firefox`, `chrome` or `chrome-beta`. Thanks, [Ben Welsh](https://github.com/palewire). [#53](https://github.com/simonw/shot-scraper/issues/53)
- New `--timeout` option for `shot-scraper shot` and `shot-scraper multi`. Thanks, [Ben Welsh](https://github.com/palewire). [#47](https://github.com/simonw/shot-scraper/pull/47)
- `shot-scraper multi` now continues to create other shots despite a timeout error, unless `--fail-on-error` is passed. Thanks, [Ryan Cheley](https://github.com/ryancheley). [#50](https://github.com/simonw/shot-scraper/issues/50)
- README now describes the `async () => { ... }` pattern for `shot-scraper javascript`, as discussed in [Extracting web page content using Readability.js and shot-scraper](https://til.simonwillison.net/shot-scraper/readability).
- New demo: [shot-scraper-template](https://github.com/simonw/shot-scraper-template), which lets you setup shot-scraper running in a GitHub repository in a single click. [See also Instantly create a GitHub repository to take screenshots of a web page](https://simonwillison.net/2022/Mar/14/shot-scraper-template/).
- New scraping demo: [scrape-hacker-news-by-domain](https://github.com/simonw/scrape-hacker-news-by-domain) uses `shot-scraper javascript` to scrape a web page. See [Scraping web pages from the command-line with shot-scraper](https://simonwillison.net/2022/Mar/14/scraping-web-pages-shot-scraper/).

- New [shot-scraper javascript](https://github.com/simonw/shot-scraper/blob/0.9/README.md#scraping-pages-using-javascript) command for executing JavaScript against a web page and returning the result to the console as JSON: [38](https://github.com/simonw/shot-scraper/issues/38)

% shot-scraper javascript datasette.io document.title
"Datasette: An open source multi-tool for exploring and publishing data"

This can be used for web scraping and data extraction. Any JavaScript errors will cause the command to return an exit code of 1, so this can also be used to run tests against a website from within a continuous integration environment such as GitHub Actions.

- The `shot-scraper pdf` and `shot-scraper accessibility` commands can both now be used with local files in addition to URLs. [37](https://github.com/simonw/shot-scraper/issues/37)
- The `output:` key is no longer required in YAML shot configuration: if omitted, an automatic filename will be used instead. [40](https://github.com/simonw/shot-scraper/issues/40)
- An empty YAML file no longer produces an error. [41](https://github.com/simonw/shot-scraper/issues/41)

- `shot-scraper` can now take screenshots of local files on disk: [35](https://github.com/simonw/shot-scraper/issues/35)

shot-scraper index.html -o index.png

- If you call `shot-scraper` on a URL with no protocol, `http://` will be assumed. Redirects will be followed:

shot-scraper datasette.io -o datasette.png

- The `shot-scraper shot` and `shot-scraper pdf` commands both now default to writing a file to disk if no filename is specified, using a name derived from the URL. If you want to write the PNG or PDF content to standard output you can do so using `-o -`. [32](https://github.com/simonw/shot-scraper/issues/32)
- New `--retina` flag for `shot-scraper shot` and `shot-scraper multi` which causes the screenshot to be taken with a device scale factor of 2. [33](https://github.com/simonw/shot-scraper/issues/33)
- `shot-scraper shot --devtools` option opens an interactive browser window with the browser developer tools enabled. [34](https://github.com/simonw/shot-scraper/issues/34)

- Now supports taking screenshots of **pages that require authentication**. [18](https://github.com/simonw/shot-scraper/issues/18)

The following command will open a browser window for the specified website, wait for you to manually authenticate and hit `<enter>` in the terminal, and then write the resulting authentication context out to `auth.json`:

shot-scraper auth https://github.com/ auth.json`

You can then take authenticated screenshots like this:

shot-scraper https://github.com/notifications \
--auth auth.json -o notifications.png

The `-a/--auth` option is also supported by the `multi`, `pdf` and `accessibility` commands.

- The `shot-scraper` command can now open a browser in which you can **interact with a page** before the screenshot is taken: [31](https://github.com/simonw/shot-scraper/issues/31)

shot-scraper https://simonwillison.net/ \
-o after-interaction.png \
--height 800 --interactive

This will output:

Hit <enter> to take the shot and close the browser window:
And after you hit <enter>...
Screenshot of 'https://simonwillison.net/' written to 'after-interaction.png'

- You can now pass **multiple CSS selectors** in order to take a screenshot of the smallest area that encompasses all of the content referenced by those selectors: [21](https://github.com/simonw/shot-scraper/issues/21)

shot-scraper https://simonwillison.net/ \
-s 'bighead' -s .overband \
-o bighead-multi-selector.png

Add `--padding 20` to include an additional 20px of padding around the specified area.

The YAML format used by `snap-shotter multi` also now supports multiple CSS selectors, which look like this:

yaml
- output: bighead-multi-selector.png
url: https://simonwillison.net/
selectors:
- "bighead"
- .overband
padding: 20


- Scripted tests can now be run using `tests/run_examples.sh` [29](https://github.com/simonw/shot-scraper/issues/29)