Sae-lens

Feature

* feat: make a neuronpedia list with features via api call (101) ([`23e680d`](https://github.com/jbloomAus/SAELens/commit/23e680d2ae03b4c375651e14feb247d3eb29e516))

Unknown

* Merge pull request 100 from jbloomAus/np_improvements

Improvements to Neuronpedia Runner ([`5118f7f`](https://github.com/jbloomAus/SAELens/commit/5118f7f7019cb3b33a26f0acc8fca55e2074202b))

* neuronpedia: save run settings to json file to avoid errors when resuming later. automatically skip batch files that already exist ([`4b5412b`](https://github.com/jbloomAus/SAELens/commit/4b5412b4c351156f20b965a4675b5448781dc951))

* skip batch file if it already exists ([`7d0e396`](https://github.com/jbloomAus/SAELens/commit/7d0e3961903fbc559bc6f7e92b497d87b5a34244))

* neuronpedia: include log sparsity threshold in skipped_indexes.json ([`5c967e7`](https://github.com/jbloomAus/SAELens/commit/5c967e7e83e27dd10bb5fbf5ba4a1291af41a16b))

Chore

* chore: enabling pythong 3.12 checks for CI ([`25526ea`](https://github.com/jbloomAus/SAELens/commit/25526ea2f72aeee77daad677cfd3555fe48d9e88))

* chore: setting up precommit to be consistent with CI ([`18e706d`](https://github.com/jbloomAus/SAELens/commit/18e706dad75f3f9bddf388f5f5c32328669749fe))

Feature

* feat: Added `tanh-relu` activation fn and input noise options (77)

* Still need to pip-install from GitHub hufy implementation.

* Added support for `tanh_sae`.

* Added notebook for loading the `tanh_sae`

* tweaking config options to be more declarating / composable

* testing adding noise to SAE forward pass

* updating notebook

---------

Co-authored-by: David Chanin <chanindavgmail.com> ([`551e94d`](https://github.com/jbloomAus/SAELens/commit/551e94de83797791e10d9c67e328031e207718c5))

Unknown

* Update proposal.md ([`6d45b33`](https://github.com/jbloomAus/SAELens/commit/6d45b33423fb308343c0b5bb0a49b12322b8c620))

* Merge pull request 96 from jbloomAus/github-templates

add templates for PR&39;s / issues ([`241a201`](https://github.com/jbloomAus/SAELens/commit/241a20195d58fbc03f33218458d43e5cbedf1745))

* add templates for PR&39;s / issues ([`74ff597`](https://github.com/jbloomAus/SAELens/commit/74ff5979e9d64b7af9fc4e0978da37857c6c8dcf))

* Merge pull request 95 from jbloomAus/load-state-dict-not-strict

Make load_state_dict use strict=False ([`4a9e274`](https://github.com/jbloomAus/SAELens/commit/4a9e2748ce34510b92328606fd5351851472eb48))

* fix accidental bug ([`c22fbbd`](https://github.com/jbloomAus/SAELens/commit/c22fbbdebc31c42cc579a1ff6433cfc73a8aeb2b))

* fix load pretrained legacy with state dict change ([`b5e97f8`](https://github.com/jbloomAus/SAELens/commit/b5e97f8b343be5c2bed36914741df9dad96361b7))

* Make load_state_dict use strict=False ([`fdf7fe9`](https://github.com/jbloomAus/SAELens/commit/fdf7fe951bf8275b7b9bee62e7c06b83570f2798))

* Merge pull request 94 from jbloomAus/update-pre-commit

chore: setting up precommit to be consistent with CI ([`6a056b7`](https://github.com/jbloomAus/SAELens/commit/6a056b7b6914f6bf6dfca5eb006f3114d4d61d11))

* Merge pull request 87 from evanhanders/old_to_new

Adds function that converts old .pt pretrained SAEs to new folder format ([`1cb1725`](https://github.com/jbloomAus/SAELens/commit/1cb17251c4a5f9d3e19c1cd4b109b58e7c6f0e7c))

* Merge pull request 93 from jbloomAus/py-312-ci

chore: enabling python 3.12 checks for CI ([`87be422`](https://github.com/jbloomAus/SAELens/commit/87be42246e9173bacd98530368e2267e8e917912))

Chore

* chore: re-enabling isort in CI (86) ([`9c44731`](https://github.com/jbloomAus/SAELens/commit/9c44731a9b7718c9f0913136ed9df42dac87c390))

Fix

* fix: pin pyzmq==26.0.1 temporarily ([`0094021`](https://github.com/jbloomAus/SAELens/commit/00940219754ddb1be6708e54cdd0ac6ed5dc3948))

* fix: typing issue, temporary ([`25cebf1`](https://github.com/jbloomAus/SAELens/commit/25cebf1e5e0630a377a5045c1b3571a5f181853f))

Unknown

* v0.5.1 ([`0ac218b`](https://github.com/jbloomAus/SAELens/commit/0ac218bf8068b8568310b40a1399f9eb3c8d992e))

* fixes string vs path typing errors ([`94f1fc1`](https://github.com/jbloomAus/SAELens/commit/94f1fc127f74dbd95ae586ff3ce4d15605403e14))

* removes unused import ([`06406b0`](https://github.com/jbloomAus/SAELens/commit/06406b096f5295aab2235247def704cddd4d3dd4))

* updates formatting for alignment with repo standards. ([`5e1f342`](https://github.com/jbloomAus/SAELens/commit/5e1f3428058d63ffa8e627f322909685ed1fdf59))

* consolidates with SAE class load_legacy function & adds test ([`0f85ded`](https://github.com/jbloomAus/SAELens/commit/0f85dede8437271724f1ec5dc79cc97829dee49b))

* adds old->new file conversion function ([`fda2b57`](https://github.com/jbloomAus/SAELens/commit/fda2b571bc303545b5714bee3c27d3f4a2e56186))

* Merge pull request 91 from jbloomAus/decoder-fine-tuning

Decoder fine tuning ([`1fc652c`](https://github.com/jbloomAus/SAELens/commit/1fc652c19e2a34172c1fd520565e9620366f565c))

* par update ([`2bb5975`](https://github.com/jbloomAus/SAELens/commit/2bb5975226807d352f2d3cf6b6dad7aefaf1b662))

* Merge pull request 89 from jbloomAus/fix_np

Enhance + Fix Neuronpedia generation / upload ([`38d507c`](https://github.com/jbloomAus/SAELens/commit/38d507c052875cbd78f8fd9dae45a658e47c2b9d))

* minor changes ([`bc766e4`](https://github.com/jbloomAus/SAELens/commit/bc766e4f7a8d472f647408b8a5cd3c6140d856b7))

* reformat run.ipynb ([`822882c`](https://github.com/jbloomAus/SAELens/commit/822882cac9c05449b7237b7d42ce17297903da2f))

* get decoder fine tuning working ([`11a71e1`](https://github.com/jbloomAus/SAELens/commit/11a71e1b95576ef6dc3dbec7eb1c76ce7ca44dfd))

* format ([`040676d`](https://github.com/jbloomAus/SAELens/commit/040676db6814c1f64171a32344e0bed40528c8f9))

* Merge pull request 88 from jbloomAus/get_feature_from_neuronpedia

FEAT: Add API for getting Neuronpedia feature ([`1666a68`](https://github.com/jbloomAus/SAELens/commit/1666a68bb7d7ee4837e03d95b203ee371ca9ea9e))

* Fix resuming from batch ([`145a407`](https://github.com/jbloomAus/SAELens/commit/145a407f8d57755301bf56c87efd4e775c59b980))

* Use original repo for sae_vis ([`1a7d636`](https://github.com/jbloomAus/SAELens/commit/1a7d636c95ef508dde8bd100ab6d9f241b0be977))

* Use correct model name for np runner ([`138d5d4`](https://github.com/jbloomAus/SAELens/commit/138d5d445878c0830c6c96a5fbe6b10a1d9644b0))

* Merge main, remove eindex ([`6578436`](https://github.com/jbloomAus/SAELens/commit/6578436891e71a0ef60fb2ed6d6a6b6279d71cbc))

* Add API for getting Neuronpedia feature ([`e78207d`](https://github.com/jbloomAus/SAELens/commit/e78207d086cb3372dc805cbb4c87b694749cd905))

Feature

* feat: Mamba support vs mamba-lens (79)

* mamba support

* added init

* added optional model kwargs

* Support transformers and mamba

* forgot one model kwargs

* failed opts

* tokens input

* hack to fix tokens, will look into fixing mambalens

* fixed checkpoint

* added sae group

* removed some comments and fixed merge error

* removed unneeded params since that issue is fixed in mambalens now

* Unneded input param

* removed debug checkpoing and eval

* added refs to hookedrootmodule

* feed linter

* added example and fixed loading

* made layer for eval change

* fix linter issues

* adding mamba-lens as optional dep, and fixing typing/linting

* adding a test for loading mamba model

* adding mamba-lens to dev for CI

* updating min mamba-lens version

* updating mamba-lens version

---------

Co-authored-by: David Chanin <chanindavgmail.com> ([`eea7db4`](https://github.com/jbloomAus/SAELens/commit/eea7db4b99098c33cd862e7e2280a32b630826bd))

Unknown

* update readme ([`440df7b`](https://github.com/jbloomAus/SAELens/commit/440df7b6c0ef55ba3d116054f81e1ee4a58f9089))

* update readme ([`3694fd2`](https://github.com/jbloomAus/SAELens/commit/3694fd2c4cc7438121e4549636508c45835a5d38))

* Fix upload skipped/dead features ([`932f380`](https://github.com/jbloomAus/SAELens/commit/932f380971ce3d431e6592c804d12f6df2b4ec78))

* Use python typer instead of shell script for neuronpedia jobs ([`b611e72`](https://github.com/jbloomAus/SAELens/commit/b611e721dd2620ab5a030cc0f6e37029c30711ca))

* Merge branch &39;main&39; into fix_np ([`cc6cb6a`](https://github.com/jbloomAus/SAELens/commit/cc6cb6a96b793e41fb91f4ebbaf3bfa5e7c11b4e))

* convert sparsity to log sparsity if needed ([`8d7d404`](https://github.com/jbloomAus/SAELens/commit/8d7d4040033fb80c5b994cdc662b0f90b8fcc7aa))

Feature

* feat: support orthogonal decoder init and no pre-decoder bias ([`ac606a3`](https://github.com/jbloomAus/SAELens/commit/ac606a3b85ac48dd800e434878b3a8bfe1838408))

Fix

* fix: sae dict bug ([`484163e`](https://github.com/jbloomAus/SAELens/commit/484163ed4c7395694d78f8a0ce1be9570a0ded2a))

* fix: session loader wasn&39;t working ([`a928d7e`](https://github.com/jbloomAus/SAELens/commit/a928d7e24224cc79eedfd5c050d02d5f22de86e8))

Unknown

* enable setting adam pars in config ([`1e53ede`](https://github.com/jbloomAus/SAELens/commit/1e53edee5cc4756283e95f824693de3c781ec532))

* fix sae dict loader and format ([`c558849`](https://github.com/jbloomAus/SAELens/commit/c558849bbf310961cd0905eab087572708272774))

* default orthogonal init false ([`a8b0113`](https://github.com/jbloomAus/SAELens/commit/a8b0113140bd2f9b97befccc8f158dace02a4810))

* Formatting ([`1e3d53e`](https://github.com/jbloomAus/SAELens/commit/1e3d53ec2b72897bfebb6065f3b530fe65d3a97c))

* Eindex required by sae_vis ([`f769e7a`](https://github.com/jbloomAus/SAELens/commit/f769e7a65ab84d4073852931a86ff3b5076eea3c))

* Upload dead feature stubs ([`9067380`](https://github.com/jbloomAus/SAELens/commit/9067380bf67b89d8b2d235944f696016286f683e))

* Make feature sparsity an argument ([`8230570`](https://github.com/jbloomAus/SAELens/commit/8230570297d68e35cb614a63abf442e4a01174d2))

* Fix buffer&34; ([`dde2481`](https://github.com/jbloomAus/SAELens/commit/dde248162b70ff4311d4182333b7cce43aed78df))

* Merge branch &39;main&39; into fix_np ([`6658392`](https://github.com/jbloomAus/SAELens/commit/66583923cd625bfc1c1ef152bc5f5beaa764b2d6))

* notebook update ([`feca408`](https://github.com/jbloomAus/SAELens/commit/feca408cf003737cd4eb529ca7fea2f77984f5c6))

* Merge branch &39;main&39; into fix_np ([`f8fb3ef`](https://github.com/jbloomAus/SAELens/commit/f8fb3efbde7fc79e6fafe2d9b3324c9f0b2a337d))

* Final fixes ([`e87788d`](https://github.com/jbloomAus/SAELens/commit/e87788d63a9b767e34e497c85a318337ab8aabb8))

* Don&39;t use buffer, fix anomalies ([`2c9ca64`](https://github.com/jbloomAus/SAELens/commit/2c9ca642b334b7a444544a4640c483229dc04c62))

Feature

* feat: add basic tutorial for training saes ([`1847280`](https://github.com/jbloomAus/SAELens/commit/18472800481dbe584e5fab8533ac47a1ee39a062))