Sae-lens

Feature

* feat: OpenAI TopK SAEs for residual stream of GPT2 Small (201)

* streamlit app

* feat: basic top-k support + oai gpt2small saes

* fix merge mistake ([`06c4302`](https://github.com/jbloomAus/SAELens/commit/06c4302eedaaa4ba95686ab6b9a49fed4652ead7))

Unknown

* prevent context size mismatch error (200) ([`76389ac`](https://github.com/jbloomAus/SAELens/commit/76389ac007b77ed639035bdaee2e7587a949a9fc))

* point gpt2 dataset path to apollo-research/monology-pile (199) ([`d3eb427`](https://github.com/jbloomAus/SAELens/commit/d3eb427c64765fc27950249d299791bfa17b6a73))

Feature

* feat: harmize activation store and pretokenize runner (181)

* eat: harmize activation store and pretokenize runner

* reverting SAE cfg back to prepend_bos

* adding a benchmark test

* adding another test

* adding list of tokenized datasets to docs

* adding a warning message about lack of pre-tokenization, and linking to SAELens docs

* fixing tests after apollo deleted sae- dataset versions

* Update training_saes.md ([`2e6a3c3`](https://github.com/jbloomAus/SAELens/commit/2e6a3c3b72e0724b24dd8ed3803f3b80a17b77d5))

Unknown

* Updating example commands ([`265687c`](https://github.com/jbloomAus/SAELens/commit/265687c09ba3c6ae090cf5a97e7f70251c0cf66c))

* Fixing test ([`389a159`](https://github.com/jbloomAus/SAELens/commit/389a15924345c17442937e98f45c8d2eb9c92b21))

* Adding script to evals.py ([`f9aa2dd`](https://github.com/jbloomAus/SAELens/commit/f9aa2ddd20c1f8c26b9181e685f04c7638511bc1))

* Moving file ([`4be5011`](https://github.com/jbloomAus/SAELens/commit/4be50115b8b2c43448557ee54ff8f0afe692d111))

* First round of evals ([`2476afb`](https://github.com/jbloomAus/SAELens/commit/2476afbffad41406840ebd5492c04acf90a0e62c))

Feature

* feat: new saes for gemma-2b-it and feature splitting on gpt2-small-layer-8 (195) ([`5cfe382`](https://github.com/jbloomAus/SAELens/commit/5cfe382d43f028c2e4f4e7cb21a1c19abb5471d0))

Feature

* feat: Support Gated-SAEs (188)

* Initial draft of encoder

* Second draft of Gated SAE implementation

* Added SFN loss implementation

* Latest modification of SFN loss training setup

* fix missing config use

* dont have special sfn loss

* add hooks and reshape

* sae error term not working, WIP

* make tests pass

* add benchmark for gated

---------

Co-authored-by: Joseph Bloom <jbloomausgmail.com> ([`232c39c`](https://github.com/jbloomAus/SAELens/commit/232c39cea709ae9c4b68b204cf027fbaab385f64))

Unknown

* fix hook z loader (194) ([`cb30996`](https://github.com/jbloomAus/SAELens/commit/cb30996cf36c80bbf6f0fd529bd27262bbce13ce))

Feature

* feat: trigger release ([`1a4663b`](https://github.com/jbloomAus/SAELens/commit/1a4663b7eadb42682586697e72192346c66cf430))

Unknown

* Performance improvements + using multiple GPUs. (189)

* fix: no grads when filling cache

* trainer should put activations on sae device

* hack to allow sae device to be specific gpu when model is on multiple devices

* add some tests (not in CI, which check multiple GPU performance

* make formatter typer happy

* make sure SAE calls move data between devices as needed ([`400474e`](https://github.com/jbloomAus/SAELens/commit/400474eaf758d57ea4090fd08e84cbdd91d55cc4))

Fix

* fix: allow settings trust_remote_code for new huggingface version (187)

* fix: allow settings trust_remote_code for new huggingface version

* default to True, not none

---------

Co-authored-by: jbloomAus <jbloomausgmail.com> ([`33a612d`](https://github.com/jbloomAus/SAELens/commit/33a612d7f694390a5a4596f7e15e0c51657634ba))