Sae-lens

Feature

* feat: Replace assert statements with exception code (400)

* replaces assert statements with exception code

* replaces assert statements with exception code in less obvious cases

* removes unnecessary if and else statements ([`324be25`](https://github.com/jbloomAus/SAELens/commit/324be2581435fffa4bc9e31a0d329aefce6cc91f))

Unknown

* Merge pull request 398 from jbloomAus/np_yaml

fix: width for llamascope 32x was incorrect ([`0617dba`](https://github.com/jbloomAus/SAELens/commit/0617dba96bf4acd64354b95e41d7abe68b6f7693))

* Update Llama Scope NP ids ([`26899cd`](https://github.com/jbloomAus/SAELens/commit/26899cdcb5f4831682162505abeb980e375888bb))

Chore

* chore: fix tokenizer typing for bos_token_id (399) ([`b3b67d6`](https://github.com/jbloomAus/SAELens/commit/b3b67d6c26d7a088f02bd78f4082ec134a6fd6a0))

* chore: Replace isort black and flake8 with ruff (393)

* replaces in cache_activations_runner.py

* replaces isort, black, adn flake8 with Ruff

* adds SIM lint rule

* fixes for CI check

* adds RET lint rule

* adds LOG lint rule

* fixes RET error

* resolves conflicts

* applies make format

* adds T20 rule

* replaces extend-select with select

* resolves conflicts

* fixes lint errors

* update .vscode/settings.json

* Revert &34;update .vscode/settings.json&34;

This reverts commit 1bb5497d7495f7fb0843bc4eb885ba90cf6b4f47.

* updates .vscode/settings.json

* adds newline ([`52dbff9`](https://github.com/jbloomAus/SAELens/commit/52dbff9d4311b873641c17cadcdc8a7f2c562269))

Feature

* feat: Save estimated norm scaling factor during checkpointing (395)

* refactor saving

* save estimated_norm_scaling_factor

* use new constant names elsewhere

* estimate norm scaling factor in `ActivationsStore` init

* fix tests

* add test

* tweaks

* safetensors path

* remove scaling factor on fold

* test scaling factor value

* format

* format

* undo silly change

* format

* save fn protocol

* make save fn static

* test which checkpoints have estimated norm scaling factor

* fix test

* fmt ([`63a15a0`](https://github.com/jbloomAus/SAELens/commit/63a15a010c3f018ae227584a0bc2866b04fe4f79))

Fix

* fix: width for llamascope 32x was incorrect ([`355691f`](https://github.com/jbloomAus/SAELens/commit/355691f5becf1b20a8a2082869fff98a2314310d))

* fix: force build ([`53180e0`](https://github.com/jbloomAus/SAELens/commit/53180e000928695748dc56787f9995f3ee35096c))

* fix: typo in pretrained yaml ([`9db9e36`](https://github.com/jbloomAus/SAELens/commit/9db9e3660f866322a756ca7f596077537a5fa25e))

Unknown

* Merge pull request 397 from jbloomAus/np_yaml

fix: typo in pretrained yaml ([`19bcb2e`](https://github.com/jbloomAus/SAELens/commit/19bcb2e3245962add858c289802cf9fb57c014b4))

Feature

* feat: Replace print with controllable logging (388)

* replaces in pretrained_sae_loaders.py

* replaces in load_model.py

* replaces in neuronpedia_integration.py

* replaces in tsea.py

* replaces in pretrained_saes.py

* replaces in cache_activations_runner.py

* replaces in activations_store.py

* replaces in training_sae.py

* replaces in upload_saes_to_huggingface.py

* replaces in sae_training_runner.py

* replaces in config.py

* fixes error for CI

---------

Co-authored-by: David Chanin <chanindavgmail.com> ([`2bcd646`](https://github.com/jbloomAus/SAELens/commit/2bcd646bf69a116d5a7df14d2fe07988539a930b))

Breaking

* feat: Cleaned up CacheActionsRunnerConfig (389)

BREAKING CHANGE: Superfluous config options have been removed

* Cleaned up CacheActionsRunnerConfig

Before `CacheActivationConfig` had a inconsistent config file for some
interopability with `LanguageModelSAERunnerConfig`. It was kind of
unclear which parameters were necessary vs redundant, and just was
fairly unclear.

Simplified to the required arguments:

- `dataset_path`: Tokenized or untokenized dataset
- `total_training_tokens`
- `model_name`
- `model_batch_size`
- `hook_name`
- `final_hook_layer`
- `d_in`

I think this scheme captures everything you need when attempting to
cache activations and makes it a lot easier to reason about.

Optional:


activation_save_path defaults to &34;activations/{dataset}/{model}/{hook_name}
shuffle=True
prepend_bos=True
streaming=True
seqpos_slice
buffer_size_gb=2 Size of each buffer. Affects memory usage and saving freq
device=&34;cuda&34; or &34;cpu&34;
dtype=&34;float32&34;
autocast_lm=False
compile_llm=True
hf_repo_id Push to hf
model_kwargs `run_with_cache`
model_from_pretrained_kwargs


* Keep compatiability with old config

- Renamed to keep values same where possible
- Moved _from_saved_activations (private api for CachedActivationRunner)
to cached_activation_runner.py
- Use properties instead of `__post_init__` ([`d81e286`](https://github.com/jbloomAus/SAELens/commit/d81e2862ce914c0b0f86c544fa8f4320c82032ac))

Fix

* fix: add missing np ([`9d26da4`](https://github.com/jbloomAus/SAELens/commit/9d26da40a35dcd335038df8724f94558a80766e0))

Unknown

* Merge pull request 387 from jbloomAus/np_yaml

fix: add missing neuronpedia yaml entries ([`deae2a7`](https://github.com/jbloomAus/SAELens/commit/deae2a7e81552b9f055baf7da1084231a3a5811c))