Sae-lens

Fix

* fix: Gated SAE Note Loading (211)

* fix: add tests, make pass

* not in ([`b083feb`](https://github.com/jbloomAus/SAELens/commit/b083feb5ffb5b7f45669403786c3c7593aa1d3ba))

Unknown

* SAETrainingRunner takes optional HFDataset (206)

* SAETrainingRunner takes optional HFDataset

* more explicit errors when the buffer is too large for the dataset

* format

* add warnings when a new dataset is added

* replace default dataset with empty string

* remove valueerror ([`2c8fb6a`](https://github.com/jbloomAus/SAELens/commit/2c8fb6aeed214ff47dccfe427eb2881aca4e6808))

Fix

* fix: pin typing-extensions version (205) ([`3f0e4fe`](https://github.com/jbloomAus/SAELens/commit/3f0e4fe9e1a353e8b9563567919734af662ab69d))

Feature

* feat: OpenAI TopK SAEs for residual stream of GPT2 Small (201)

* streamlit app

* feat: basic top-k support + oai gpt2small saes

* fix merge mistake ([`06c4302`](https://github.com/jbloomAus/SAELens/commit/06c4302eedaaa4ba95686ab6b9a49fed4652ead7))

Unknown

* prevent context size mismatch error (200) ([`76389ac`](https://github.com/jbloomAus/SAELens/commit/76389ac007b77ed639035bdaee2e7587a949a9fc))

* point gpt2 dataset path to apollo-research/monology-pile (199) ([`d3eb427`](https://github.com/jbloomAus/SAELens/commit/d3eb427c64765fc27950249d299791bfa17b6a73))

Feature

* feat: harmize activation store and pretokenize runner (181)

* eat: harmize activation store and pretokenize runner

* reverting SAE cfg back to prepend_bos

* adding a benchmark test

* adding another test

* adding list of tokenized datasets to docs

* adding a warning message about lack of pre-tokenization, and linking to SAELens docs

* fixing tests after apollo deleted sae- dataset versions

* Update training_saes.md ([`2e6a3c3`](https://github.com/jbloomAus/SAELens/commit/2e6a3c3b72e0724b24dd8ed3803f3b80a17b77d5))

Unknown

* Updating example commands ([`265687c`](https://github.com/jbloomAus/SAELens/commit/265687c09ba3c6ae090cf5a97e7f70251c0cf66c))

* Fixing test ([`389a159`](https://github.com/jbloomAus/SAELens/commit/389a15924345c17442937e98f45c8d2eb9c92b21))

* Adding script to evals.py ([`f9aa2dd`](https://github.com/jbloomAus/SAELens/commit/f9aa2ddd20c1f8c26b9181e685f04c7638511bc1))

* Moving file ([`4be5011`](https://github.com/jbloomAus/SAELens/commit/4be50115b8b2c43448557ee54ff8f0afe692d111))

* First round of evals ([`2476afb`](https://github.com/jbloomAus/SAELens/commit/2476afbffad41406840ebd5492c04acf90a0e62c))

Feature

* feat: new saes for gemma-2b-it and feature splitting on gpt2-small-layer-8 (195) ([`5cfe382`](https://github.com/jbloomAus/SAELens/commit/5cfe382d43f028c2e4f4e7cb21a1c19abb5471d0))

Feature

* feat: Support Gated-SAEs (188)

* Initial draft of encoder

* Second draft of Gated SAE implementation

* Added SFN loss implementation

* Latest modification of SFN loss training setup

* fix missing config use

* dont have special sfn loss

* add hooks and reshape

* sae error term not working, WIP

* make tests pass

* add benchmark for gated

---------

Co-authored-by: Joseph Bloom <jbloomausgmail.com> ([`232c39c`](https://github.com/jbloomAus/SAELens/commit/232c39cea709ae9c4b68b204cf027fbaab385f64))

Unknown

* fix hook z loader (194) ([`cb30996`](https://github.com/jbloomAus/SAELens/commit/cb30996cf36c80bbf6f0fd529bd27262bbce13ce))