Sae-lens

Fix

* fix: Fix issues with resumption testing (144)

* fix always-true comparison in train context testing

* set default warmup steps to zero

* remove unused type attribute from L1Scheduler

* update training tests to use real context builder

* add docstring for build_train_ctx ([`085d04f`](https://github.com/jbloomAus/SAELens/commit/085d04f7e57e3819810b18e12b011adc8c7f2ba1))

Fix

* fix: hardcoded mps device in ckrk attn saes (141) ([`eba3f4e`](https://github.com/jbloomAus/SAELens/commit/eba3f4e54ad6a02553f0ed2b575b0547df68a200))

Unknown

* feature: run saelens on AWS with one command (138)

* Ansible playbook for automating caching activations and training saes

* Add automation

* Fix example config

* Fix bugs with ansible mounting s3

* Reorg, more automation, Ubuntu instead of Amazon Linux

* More automation

* Train SAE automation

* Train SAEs and readme

* fix gitignore

* Fix automation config bugs, clean up paths

* Fix shutdown time, logs ([`13de52a`](https://github.com/jbloomAus/SAELens/commit/13de52a5e12fd275ca8601aa22fd5ec66a5c6e9a))

* Gpt 2 sweep (140)

* sweep settings for gpt2-small

* get model string right

* fix some comments that don&39;t apply now

* formatting fix ([`4cb270b`](https://github.com/jbloomAus/SAELens/commit/4cb270b7680585c5758910dfeafa727185ac88b9))

* Remove cuda cache emptying in evals.py (139) ([`bdef2cf`](https://github.com/jbloomAus/SAELens/commit/bdef2cf0e4ad3e3070f9be1c46a1adf13094eb13))

Chore

* chore: remove use_deterministic_algorithms=True since it causes cuda errors (137) ([`1a3bedb`](https://github.com/jbloomAus/SAELens/commit/1a3bedbc3192ca919fc0716ce52d06f060ad2550))

Feature

* feat: Hooked toy model (134)

* adds initial re-implementation of toy models

* removes instance dimension from toy models

* fixing up minor nits and adding more tests

---------

Co-authored-by: David Chanin <chanindavgmail.com> ([`03aa25c`](https://github.com/jbloomAus/SAELens/commit/03aa25c6e8589c1eb9a6b94911e6d77187d6bef7))

Breaking

* feat: rename batch sizes to give informative units (133)

BREAKING CHANGE: renamed batch sizing config params

* renaming batch sizes to give units

* changes in notebooks

* missed one!

---------

Co-authored-by: David Chanin <chanindavgmail.com> ([`cc78e27`](https://github.com/jbloomAus/SAELens/commit/cc78e277ae639f57df389e61899278919c16d993))

Chore

* chore: tools to make tests more deterministic (132) ([`2071d09`](https://github.com/jbloomAus/SAELens/commit/2071d096c46b0b532e2d99381b300c3c64071747))

* chore: Make tutorial notebooks work in Google Colab (120)

Co-authored-by: David Chanin <chanindavgmail.com> ([`007141e`](https://github.com/jbloomAus/SAELens/commit/007141e67a447f8dfe8d6797a6fba0fc4fce61bd))

Chore

* chore: closing &34; in docs (130) ([`5154d29`](https://github.com/jbloomAus/SAELens/commit/5154d29498a480c4e7ddc9edc9effd30cecbeda7))

Feature

* feat: Add model_from_pretrained_kwargs as config parameter (122)

* add model_from_pretrained_kwargs config parameter to allow full control over model used to extract activations from. Update tests to cover new cases

* tweaking test style

---------

Co-authored-by: David Chanin <chanindavgmail.com> ([`094b1e8`](https://github.com/jbloomAus/SAELens/commit/094b1e8250c1ae6cd3fc8336e09075d61ce967d1))

Feature

* feat: Add torch compile (129)

* Surface of eval batches and of eval sequences

* fix formatting

* config changes

* add compilation to lm_runner.py

* remove accidental print statement

* formatting fix ([`5c41336`](https://github.com/jbloomAus/SAELens/commit/5c41336853beac6bbe2105fefc55c746e3e2e61f))

* feat: Change eval batch size (128)

* Surface of eval batches and of eval sequences

* fix formatting

* fix print statement accidentally left in ([`758a50b`](https://github.com/jbloomAus/SAELens/commit/758a50b073777028cd0dabcc50049798c2fcd68f))