Sae-lens

Chore

* chore: fixing whitespace so docs render as list not paragraph (374) ([`156ddc9`](https://github.com/jbloomAus/SAELens/commit/156ddc9fbdc92217147257886f38a2d719e45bff))

* chore: add codecov.yaml and exclude legacy files (372) ([`aa98caf`](https://github.com/jbloomAus/SAELens/commit/aa98caf9658a99fcb5c4295719110475b46f5862))

Fix

* fix: add neuronpedia ids for llamascope ([`23b4912`](https://github.com/jbloomAus/SAELens/commit/23b491246426e03cae522282c1d491696bfe6433))

Unknown

* Merge pull request 375 from jbloomAus/add_np_llamascope

fix: add neuronpedia ids for llamascope ([`60542fa`](https://github.com/jbloomAus/SAELens/commit/60542fa6f5c88ee9d08fc056f984863c9e07de66))

* Merge pull request 371 from jbloomAus/fix-llamascope-details

fixed llamascope sae names and loader ([`fecfe5d`](https://github.com/jbloomAus/SAELens/commit/fecfe5df8d478d1029b316071b3e05d08a5d4cf5))

* fixed llamascope sae names and loader ([`8f6bcb0`](https://github.com/jbloomAus/SAELens/commit/8f6bcb0f54345de7d1416da1b84efd27ddeba6ff))

* Merge pull request 369 from Hzfinfdu/main

Add Llama Scope SAEs & improvements to evaluating ce scores. ([`a1546e6`](https://github.com/jbloomAus/SAELens/commit/a1546e6a9eaa32fe3c2cad0301e5bbb0946b9000))

* fix format for PR ([`1443b58`](https://github.com/jbloomAus/SAELens/commit/1443b58c45c314c803cd54af4a54ef96b373fdc2))

* feature(evals): mask ignore_tokens in replacement hooks for evaluation ([`ae67eaa`](https://github.com/jbloomAus/SAELens/commit/ae67eaab3173cf028e22f242b0f56d9413277b23))

Fix

* fix: fixing type errors after bad merge ([`4a08d0d`](https://github.com/jbloomAus/SAELens/commit/4a08d0d4126a183a7516146de798fcc2030d40fa))

* fix: only scale sparsity by dec norm if specified in the config (365) ([`ceb2d3f`](https://github.com/jbloomAus/SAELens/commit/ceb2d3f8950fd0d0ed00a394fff56bc92aa93821))

Chore

* chore: updating training docs with tips / jumprelu (366)

* chore: updating training docs with tips / jumprelu

* fixing missing space char ([`f739500`](https://github.com/jbloomAus/SAELens/commit/f73950052fe96e4cd07453b3393722b3f0c5da62))

Feature

* feat: Support arbitrary huggingface causal LM models (226)

* adding load_model helper for huggingface causal LM models

* polishing huggingface integration

* adding more tests

* updating docs

* tweaking docstrings

* perf fix: dont calculate loss by default

* better handling of HF tuple outputs

* fixing test

* changes from CR

* fixing default model params for huggingface models

* move hf model to device on load ([`044d4be`](https://github.com/jbloomAus/SAELens/commit/044d4be8b75f487da6f278adda657a6dcb7dbe7c))

Performance

* perf: faster cleanup of datasets when caching activations (367)

previously I used dataset.save_to_disk to write the final dataset, but
this can be slow. Instead I manually move the shards to the standard hf
format which allows us not to resave the entire dataset ([`a3663b7`](https://github.com/jbloomAus/SAELens/commit/a3663b79b4889f58e402790afb5701879392279c))

Chore

* chore: adding &39;Load this SAE&39; popup to docs table (362) ([`1866aa7`](https://github.com/jbloomAus/SAELens/commit/1866aa75350b18d48c4f382fd395e68c0b098574))

* chore: more flexible training losses (357)

* retun and log a dict from train step

* updating trainer loss pbar

* avoid unnecessary gpu sync

* fixing tests

* adding logging for unnormalized l1 loss ([`0c1179c`](https://github.com/jbloomAus/SAELens/commit/0c1179cc668d8ccea2cd8b4653ccc9ceabadec19))

Feature

* feat: adding a CLI training runner (359) ([`998c277`](https://github.com/jbloomAus/SAELens/commit/998c2779ebd4dda2375cc1d1f24f77889ec2cafc))

Unknown

* add support for Llama Scope SAEs ([`aaf2f29`](https://github.com/jbloomAus/SAELens/commit/aaf2f29bc45af69d6af791758b550cd38b063036))

Chore

* chore: Update training_a_sparse_autoencoder.ipynb (358)

Changed &34;She lived in a big, happy little girl.&34; to &34;She lived in a big, happy little town.&34; ([`b8703fe`](https://github.com/jbloomAus/SAELens/commit/b8703fe8332b6eb6c49df778f6550c59d2276458))

Fix

* fix: load the same config from_pretrained and get_sae_config (361)

* fix: load the same config from_pretrained and get_sae_config

* merge neuronpedia_id into get_sae_config

* fixing test ([`8e09458`](https://github.com/jbloomAus/SAELens/commit/8e094581c4772e33ec4577349ed0d02c6c90ed27))

Feature

* feat: Support training JumpReLU SAEs (352)

* adds JumpReLU logic to TrainingSAE

* adds unit tests for JumpReLU

* changes classes to match tutorial

* replaces bandwidth constant with param

* re-add logic to JumpReLU logic to TrainingSAE

* adds TrainingSAE.save_model()

* changes threshold to match paper

* add tests for TrainingSAE when archicture is jumprelu

* adds test for SAE.load_from_pretrained() for JumpReLU

* removes code causing test to fail

* renames initial_threshold to threshold

* removes setattr()

* adds test for TrainingSAE.save_model()

* renames threshold to jumprelu_init_threshold

* adds jumprelu_bandwidth

* removes default value for jumprelu_init_threshold downstream

* replaces zero tensor with None in Step.backward()

* adds jumprelu to architecture type ([`0b56d03`](https://github.com/jbloomAus/SAELens/commit/0b56d035ce0fa12722d62cc1bc559bd4fd35e9f3))