Sae-lens

Latest version: v3.8.0

Safety actively analyzes 642295 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 3 of 7

3.1.0

Feature

* feat: add w_dec_norm folding (167)

* feat: add w_dec_norm folding

* format ([`f1908a3`](https://github.com/jbloomAus/SAELens/commit/f1908a39b3d78a03288ca721aa18fc5bfbf9b83e))

Unknown

* Fixed typo in Hooked_SAE_Transformer_Demo.ipynb preventing Open in Colab badge from working (166)

Minor typo in file name was preventing Hooked_SAE_Transformer_Demo.ipynb &34;Open in Colab&34; badge from working. ([`4850b16`](https://github.com/jbloomAus/SAELens/commit/4850b16a17c08ef39a8df74a5f4df5074395474b))

* Fix hook z training reshape bug (165)

* remove file duplicate

* fix: hook-z evals working, and reshaping mode more explicit ([`0550ae3`](https://github.com/jbloomAus/SAELens/commit/0550ae3defe778e08a050faff5e1345aee6de1a4))

3.0.0

Breaking

* feat: refactor SAE code

BREAKING CHANGE: renamed and re-implemented paths ([`3c67666`](https://github.com/jbloomAus/SAELens/commit/3c6766604f5b2079206e0c073e75a72c67a76f43))

Unknown

* major: trigger release

BREAKING CHANGE: https://python-semantic-release.readthedocs.io/en/latest/commit-parsing.html#commit-parser-angular

BREAKING CHANGE: ([`fac8533`](https://github.com/jbloomAus/SAELens/commit/fac8533be338dcacbae0045ab0d7a7396c630aa8))

* major: trigger release

BREAKING CHANGE: trigger release (apparently we need a newline) ([`90ed2c2`](https://github.com/jbloomAus/SAELens/commit/90ed2c296fb65b6e1935435690d8cddb007ce04b))

* BREAKING CHANGE: Quality of Life Refactor of SAE Lens adding SAE Analysis with HookedSAETransformer and some other breaking changes. (162)

* move HookedSAETransformer from TL

* add tests

* move runners one level up

* fix docs name

* trainer clean up

* create training sae, not fully seperate yet

* remove accidentally commited notebook

* commit working code in the middle of refactor, more work to do

* don&39;t use act layers plural

* make tutorial not use the activation store

* moved this file

* move import of toy model runner

* saes need to store at least enough information to run them

* further refactor and add tests

* finish act store device rebase

* fix config type not caught by test

* partial progress, not yet handling error term for hooked sae transformer

* bring tests in line with trainer doing more work

* revert some of the simplification to preserve various features, ghost grads, noising

* hooked sae transformer is working

* homogenize configs

* re-enable sae compilation

* remove old file that doesn&39;t belong

* include normalize activations in base sae config

* make sure tutorial works

* don&39;t forget to update pbar

* rename sparse autoencoder to sae for brevity

* move non-training specific modules out of training

* rename to remove _point

* first steps towards better docs

* final cleanup

* have ci use same test coverage total as make check-ci

* clean up docs a bit

---------

Co-authored-by: ckkissane <67170576+ckkissaneusers.noreply.github.com> ([`e4eaccc`](https://github.com/jbloomAus/SAELens/commit/e4eaccc87b277a42d463624656a3548ead0db359))

* Move activation store to cpu (159)

* add act store device to config

* fix serialisation issue with device

* fix accidental hardcoding of a device

* test activations get moved correctly

* fix issue with test cacher that shared state

* add split store & model test + fix failure

* clarify comment

* formatting fixes ([`eb9489a`](https://github.com/jbloomAus/SAELens/commit/eb9489a2dd11fe4841857309dcc369e98a6b9360))

* Refactor training (158)

* turn training runner into a class

* make a trainer class

* further refactor

* update runner call

* update docs ([`72179c8`](https://github.com/jbloomAus/SAELens/commit/72179c8336fcdb5e159ddca930af99700362e377))

* Enable autocast for LM activation creation (157)

* add LM autocasting

* add script to test autocast performance

* format fix

* update autocast demo script ([`cf94845`](https://github.com/jbloomAus/SAELens/commit/cf94845129f0e2d0bbe5135d90797a03611e983c))

* gemma 2b sae resid post 12. fix ghost grad print ([`2a676b2`](https://github.com/jbloomAus/SAELens/commit/2a676b210832e789dbb80f33b2d8f747a7209e0f))

* don&39;t hardcode hook ([`a10283d`](https://github.com/jbloomAus/SAELens/commit/a10283de5b402cbac9c2afbd6263b9e5798f9e1c))

* add mlp out SAEs to from pretrained ([`ee9291e`](https://github.com/jbloomAus/SAELens/commit/ee9291eae91908c398377b199bb9e3b33a5a2622))

* remove resuming ability, keep resume config but complain if true (156) ([`64e4dcd`](https://github.com/jbloomAus/SAELens/commit/64e4dcd3fe142cee751a348f9ed581edf2a6e3f0))

* Add notebook to transfer W&B models to HF (154)

hard to check this works quickly but assuming it does. ([`91239c1`](https://github.com/jbloomAus/SAELens/commit/91239c1c6e0abd06aea3aa7669fc5b56adc6e792))

* Remove sae parallel training, simplify code (155)

* remove sae parallel training, simplify code
* remove unused import
* remove accidental inclusion of file

(not tagging this as breaking since we&39;re do a new major release this week and I don&39;t want to keep bumping the major version) ([`f445fdf`](https://github.com/jbloomAus/SAELens/commit/f445fdfc823cb6be8b1910a28a89c8bd20661be8))

* Update pretrained_saes.yaml ([`37fb150`](https://github.com/jbloomAus/SAELens/commit/37fb15083a5427894b65de9654272e99291ce46a))

* Ansible: update incorrect EC2 quota request link ([`432c7e1`](https://github.com/jbloomAus/SAELens/commit/432c7e1fd7e0ea64fcba5941e36d9740c2c58a07))

* Merge pull request 153 from jbloomAus/ansible_dev

Ansible: dev only mode ([`51d2175`](https://github.com/jbloomAus/SAELens/commit/51d2175d05ce99a83da7b29210e725163a578c1a))

* Ansible: dev only mode ([`027460f`](https://github.com/jbloomAus/SAELens/commit/027460f48819f7953754406f5cc0499a08ed4ebc))

* feature: add gemma-2b bootleg saes (152) ([`b9b7e32`](https://github.com/jbloomAus/SAELens/commit/b9b7e32562a1c48003671464f0ed5084d3541e97))

2.1.3

Fix

* fix: Fix normalisation (150)

* fix GPT2 sweep settings to use correct dataset

* add gpt2 small block sweep to check norm

* larger buffer + more evals

* fix activation rescaling so normalisation works

* formatting fixes ([`9ce0fe4`](https://github.com/jbloomAus/SAELens/commit/9ce0fe4747ad31be5570baa7cf31714374c98e10))

Unknown

* Fix checkpointing of training state that includes a compiled SAE (143)

* Adds state_dict to L1Scheduler

* investigating test failure

* fix: Fix issues with resumption testing (144)

* fix always-true comparison in train context testing

* set default warmup steps to zero

* remove unused type attribute from L1Scheduler

* update training tests to use real context builder

* add docstring for build_train_ctx

* 2.1.2

Automatically generated by python-semantic-release

* Adds state_dict to L1Scheduler

* investigating test failure

---------

Co-authored-by: github-actions <github-actionsgithub.com> ([`2f8c4e1`](https://github.com/jbloomAus/SAELens/commit/2f8c4e17316658b14dd3bc9d1f7e50cea36b0db4))

* fix GPT2 sweep settings to use correct dataset (147)

* fix GPT2 sweep settings to use correct dataset

* add gpt2 small block sweep to check norm

* larger buffer + more evals

---------

Co-authored-by: Joseph Bloom <69127271+jbloomAususers.noreply.github.com> ([`448d911`](https://github.com/jbloomAus/SAELens/commit/448d911e803aaa051d70e5f532933f71dcb72be8))

* Pretokenize runner (148)

* feat: adding a pretokenize runner

* rewriting pretokenization based on feedback ([`f864178`](https://github.com/jbloomAus/SAELens/commit/f8641783e48cc01f7184b1c91ddc39994afd4f4b))

* Fix config files for Ansible ([`ec70cea`](https://github.com/jbloomAus/SAELens/commit/ec70cea88fbad149f45472dae7bfe7be56351b60))

* Pin Ansible config example to a specific version, update docs (142)

* Pin Ansible config example to a specific version, update docs

* Allow running cache acts or train sae separately. Update README

* Update readme ([`41785ae`](https://github.com/jbloomAus/SAELens/commit/41785ae31dc826ac99a142eca65b05d0e57b5ce1))

2.1.2

Fix

* fix: Fix issues with resumption testing (144)

* fix always-true comparison in train context testing

* set default warmup steps to zero

* remove unused type attribute from L1Scheduler

* update training tests to use real context builder

* add docstring for build_train_ctx ([`085d04f`](https://github.com/jbloomAus/SAELens/commit/085d04f7e57e3819810b18e12b011adc8c7f2ba1))

2.1.1

Fix

* fix: hardcoded mps device in ckrk attn saes (141) ([`eba3f4e`](https://github.com/jbloomAus/SAELens/commit/eba3f4e54ad6a02553f0ed2b575b0547df68a200))

Unknown

* feature: run saelens on AWS with one command (138)

* Ansible playbook for automating caching activations and training saes

* Add automation

* Fix example config

* Fix bugs with ansible mounting s3

* Reorg, more automation, Ubuntu instead of Amazon Linux

* More automation

* Train SAE automation

* Train SAEs and readme

* fix gitignore

* Fix automation config bugs, clean up paths

* Fix shutdown time, logs ([`13de52a`](https://github.com/jbloomAus/SAELens/commit/13de52a5e12fd275ca8601aa22fd5ec66a5c6e9a))

* Gpt 2 sweep (140)

* sweep settings for gpt2-small

* get model string right

* fix some comments that don&39;t apply now

* formatting fix ([`4cb270b`](https://github.com/jbloomAus/SAELens/commit/4cb270b7680585c5758910dfeafa727185ac88b9))

* Remove cuda cache emptying in evals.py (139) ([`bdef2cf`](https://github.com/jbloomAus/SAELens/commit/bdef2cf0e4ad3e3070f9be1c46a1adf13094eb13))

2.1.0

Chore

* chore: remove use_deterministic_algorithms=True since it causes cuda errors (137) ([`1a3bedb`](https://github.com/jbloomAus/SAELens/commit/1a3bedbc3192ca919fc0716ce52d06f060ad2550))

Feature

* feat: Hooked toy model (134)

* adds initial re-implementation of toy models

* removes instance dimension from toy models

* fixing up minor nits and adding more tests

---------

Co-authored-by: David Chanin <chanindavgmail.com> ([`03aa25c`](https://github.com/jbloomAus/SAELens/commit/03aa25c6e8589c1eb9a6b94911e6d77187d6bef7))

Page 3 of 7

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.