Transformer-lens

TransformerLens officially has a 2.0! The HookedSAETransformer has been removed from TransformerLens in favor of the implementation in SAELens. Along with that, a lot of cumulative changes have been added. TransformerLens also has its first official development road map! To see full details, please view the [release announcement](https://transformerlensorg.github.io/TransformerLens/content/news/release-2.0.html) for the development roadmap, changes coming to contributors, and a few more notes on this release.

What's Changed
* Refactor components by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/563
* added convenience function for unwrapping config to replace commonly … by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/571
* unwrapped config by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/577
* Refactor integration tests by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/576
* Add Mistral 7B v0.2 Instruct by fakerybakery in https://github.com/TransformerLensOrg/TransformerLens/pull/579
* Add support for Phi-3 by slash3g in https://github.com/TransformerLensOrg/TransformerLens/pull/573
* Revert "Add Mistral 7B v0.2 Instruct" by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/586
* Interactive neuroscope ci by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/589
* removed Hooked SAE by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/600

Nice little update to fix a bug someone found, and added support for ai-forever models.

What's Changed
* Add support for ai-forever/mGPT model by SeuperHakkerJa in https://github.com/TransformerLensOrg/TransformerLens/pull/606
* moved enable hook functionality to separate functions and tested new functions by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/613


**Full Changelog**: https://github.com/TransformerLensOrg/TransformerLens/compare/v1.18.0...v1.19.0

* More pytest fixtures by bmillwood in https://github.com/TransformerLensOrg/TransformerLens/pull/609
* (v3) Draft PR: add Pyright static typing to hook_points.py 590 by starship006 in https://github.com/TransformerLensOrg/TransformerLens/pull/607
* v1.19 by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/614
* add n k v heads to model properties table by anthonyduong9 in https://github.com/TransformerLensOrg/TransformerLens/pull/610
* fixed format by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/616
* Add tests for hook point add hook by anthonyduong9 in https://github.com/TransformerLensOrg/TransformerLens/pull/617
* added release blog by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/618
* Fix llama demos by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/619
* added news link by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/620

Very important release for those using Gemma models. A recent upstream change caused the TransformerLens implementation to become outdated. This release fixes that issue, and includes a number of cumulative changes, and bug fixes. The only API change in this release is that you can now set override `trust_remote_code` in the function `from_pretrained`. Thanks to all who contributed to this release!

What's Changed
* reworked CI to publish code coverage report by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/559
* Resolve SAE CI Test failures by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/560
* Ci coverage location by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/561
* Ci full coverage by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/562
* moved coverage report download by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/564
* Revert "moved coverage report download (564)" by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/565
* Othello ci by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/567
* moved report to static section by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/566
* Fix broken HookedSAETransformer demo links by ckkissane in https://github.com/TransformerLensOrg/TransformerLens/pull/572
* Fix Pos Slice Issue by hannamw in https://github.com/TransformerLensOrg/TransformerLens/pull/578
* Hf secret by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/552
* updated pull reqeust template to account for new dev branch by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/581
* updated PR template to add a note about merging from different branches by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/583
* updated repo URL throughout the project by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/580
* Fix docs badge in README by ArthurConmy in https://github.com/TransformerLensOrg/TransformerLens/pull/585
* added debug step by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/568
* Update Gemma to reflect upstream HF changes by cmathw in https://github.com/TransformerLensOrg/TransformerLens/pull/596
* allow user to force trust_remote_code=true via from_pretrained kwargs by Butanium in https://github.com/TransformerLensOrg/TransformerLens/pull/597


**Full Changelog**: https://github.com/TransformerLensOrg/TransformerLens/compare/v1.17.0...v1.18.0

What's Changed
* HookedSAETransformer by ckkissane in https://github.com/neelnanda-io/TransformerLens/pull/536


**Full Changelog**: https://github.com/neelnanda-io/TransformerLens/compare/v1.16.0...v1.17.0

Lots of feature additions (thanks joelburget for Llama support, and sheikheddy for Llama-2-70b-chat-hf support!), and also a very helpful bugfix from wesg52. Thanks to all contributors, especially new contributors!

What's Changed
* Add support for Llama-2-70b-chat-hf by sheikheddy in https://github.com/neelnanda-io/TransformerLens/pull/525
* Update loading_from_pretrained.py by jbloomAus in https://github.com/neelnanda-io/TransformerLens/pull/529
* Bugfix: pytest import by tkukurin in https://github.com/neelnanda-io/TransformerLens/pull/532
* Remove non-existing parameter from decompose_resid() documentation by VasilGeorgiev39 in https://github.com/neelnanda-io/TransformerLens/pull/504
* Add `overload` to `FactoredMatrix.__{,r}matmul__` by JasonGross in https://github.com/neelnanda-io/TransformerLens/pull/512
* Improve documentation for abstract attribute by Felhof in https://github.com/neelnanda-io/TransformerLens/pull/508
* Add pos_slice to run_with_cache by VasilGeorgiev39 in https://github.com/neelnanda-io/TransformerLens/pull/465
* Add Support for Yi-6B and Yi-34B by collingray in https://github.com/neelnanda-io/TransformerLens/pull/494
* updated docs to account for additional test suites by bryce13950 in https://github.com/neelnanda-io/TransformerLens/pull/533
* Bugfix: remove redundant assert checks by tkukurin in https://github.com/neelnanda-io/TransformerLens/pull/534
* Speed up !pip install transformer-lens in colab by pavanyellow in https://github.com/neelnanda-io/TransformerLens/pull/510
* Add Xavier and Kaiming Initializations by Chanlaw in https://github.com/neelnanda-io/TransformerLens/pull/537
* chore: fixing type errors and enabling mypy by chanind in https://github.com/neelnanda-io/TransformerLens/pull/516
* Add Mixtral by collingray in https://github.com/neelnanda-io/TransformerLens/pull/521
* Standardize black line length to 100, in line with other project settings by Chanlaw in https://github.com/neelnanda-io/TransformerLens/pull/538
* Refactor hook_points by VasilGeorgiev39 in https://github.com/neelnanda-io/TransformerLens/pull/505
* Fix split_qkv_input for grouped query attention by wesg52 in https://github.com/neelnanda-io/TransformerLens/pull/520
* locked attribution patching to 1.1.1 by bryce13950 in https://github.com/neelnanda-io/TransformerLens/pull/541
* Demo no position fix by bryce13950 in https://github.com/neelnanda-io/TransformerLens/pull/544
* Othello colab fix by bryce13950 in https://github.com/neelnanda-io/TransformerLens/pull/545
* Fixed Santa Coder demo by bryce13950 in https://github.com/neelnanda-io/TransformerLens/pull/546
* Hf token auth by bryce13950 in https://github.com/neelnanda-io/TransformerLens/pull/550
* Fixed device being set to cpu:0 instead of cpu by Butanium in https://github.com/neelnanda-io/TransformerLens/pull/551
* Add support for Llama 3 (and Llama-2-70b-hf) by joelburget in https://github.com/neelnanda-io/TransformerLens/pull/549
* Loading of huggingface 4-bit quantized Llama by coolvision in https://github.com/neelnanda-io/TransformerLens/pull/486
* removed deuplicate rearrange block by bryce13950 in https://github.com/neelnanda-io/TransformerLens/pull/555
* Bert demo ci by bryce13950 in https://github.com/neelnanda-io/TransformerLens/pull/556

New Contributors
* sheikheddy made their first contribution in https://github.com/neelnanda-io/TransformerLens/pull/525
* tkukurin made their first contribution in https://github.com/neelnanda-io/TransformerLens/pull/532
* VasilGeorgiev39 made their first contribution in https://github.com/neelnanda-io/TransformerLens/pull/504
* JasonGross made their first contribution in https://github.com/neelnanda-io/TransformerLens/pull/512
* pavanyellow made their first contribution in https://github.com/neelnanda-io/TransformerLens/pull/510
* Chanlaw made their first contribution in https://github.com/neelnanda-io/TransformerLens/pull/537
* chanind made their first contribution in https://github.com/neelnanda-io/TransformerLens/pull/516
* wesg52 made their first contribution in https://github.com/neelnanda-io/TransformerLens/pull/520
* Butanium made their first contribution in https://github.com/neelnanda-io/TransformerLens/pull/551
* coolvision made their first contribution in https://github.com/neelnanda-io/TransformerLens/pull/486

**Full Changelog**: https://github.com/neelnanda-io/TransformerLens/compare/v1.15.0...v1.16.0