Transformer-lens

LLaMA 3.3 support! This release also includes a handful of usability improvements.

What's Changed
* Set prepend_bos to false by default for Qwen models by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/815
* Throw error when using attn_in with grouped query attention by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/810
* Feature llama 33 by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/826


**Full Changelog**: https://github.com/TransformerLensOrg/TransformerLens/compare/v2.10.0...v2.11.0

Huge update! This is likely going to be the last big 2.x update. This update greatly improves model implementation accuracy, and adds some of the newer Qwen models.

What's Changed
* Remove einsum in forward pass in AbstractAttention by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/783
* Colab compatibility bug fixes by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/794
* Remove einsum usage from create_alibi_bias function by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/781
* Actions token access by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/797
* Remove einsum in apply_causal_mask in abstract_attention.py by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/782
* clarified arguments a bit for hook_points by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/799
* Remove einsum in logit_attrs in ActivationCache by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/788
* Remove einsum in compute_head_results in ActivationCache by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/789
* Remove einsum usage in refactor_factored_attn_matrices in HookedTransformer by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/791
* Remove einsum usage in _get_w_in_matrix in SVDInterpreter by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/792
* Remove einsum usage in forward function of BertMLMHead by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/793
* Set default_prepend_bos to False in Bloom model configuration by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/806
* Remove einsum in complex_attn_linear by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/790
* Add a demo of collecting activations from a single location in the model. by adamkarvonen in https://github.com/TransformerLensOrg/TransformerLens/pull/807
* Add support for Qwen_with_Questions by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/811
* Added support for Qwen2.5 by israel-adewuyi in https://github.com/TransformerLensOrg/TransformerLens/pull/809
* Updated devcontainers to use python3.11 by jonasrohw in https://github.com/TransformerLensOrg/TransformerLens/pull/812

New Contributors
* israel-adewuyi made their first contribution in https://github.com/TransformerLensOrg/TransformerLens/pull/809
* jonasrohw made their first contribution in https://github.com/TransformerLensOrg/TransformerLens/pull/812

**Full Changelog**: https://github.com/TransformerLensOrg/TransformerLens/compare/v2.9.1...v2.10.0

Minor dependency change to address a change in an outside dependency

What's Changed
* added typeguard dependency by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/786


**Full Changelog**: https://github.com/TransformerLensOrg/TransformerLens/compare/v2.9.0...v2.9.1

Lot's of accuracy improvements! A number of models are behaving closer to how they behave in Transformers, and a new internal configuration has been added to allow for more ease of use!

What's Changed
* fix the bug that attention_mask and past_kv_cache cannot work together by yzhhr in https://github.com/TransformerLensOrg/TransformerLens/pull/772
* Set prepend_bos to false by default for Bloom model family by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/775
* Fix that if use_past_kv_cache is set to True models from the Bloom family produce weird outputs. by degenfabian in https://github.com/TransformerLensOrg/TransformerLens/pull/777

New Contributors
* yzhhr made their first contribution in https://github.com/TransformerLensOrg/TransformerLens/pull/772
* degenfabian made their first contribution in https://github.com/TransformerLensOrg/TransformerLens/pull/775

**Full Changelog**: https://github.com/TransformerLensOrg/TransformerLens/compare/v2.8.1...v2.9.0

New notebook for comparing models, and bug fix with dealing with newer LLaMA models!

What's Changed
* Logit comparator tool by curt-tigges in https://github.com/TransformerLensOrg/TransformerLens/pull/765
* Add support for NTK-by-Part Rotary Embedding & set correct rotary base for Llama-3.1 series by Hzfinfdu in https://github.com/TransformerLensOrg/TransformerLens/pull/764

New Contributors
* Hzfinfdu made their first contribution in https://github.com/TransformerLensOrg/TransformerLens/pull/764

**Full Changelog**: https://github.com/TransformerLensOrg/TransformerLens/compare/v2.8.0...v2.8.1

What's Changed
* add transformer diagram by akozlo in https://github.com/TransformerLensOrg/TransformerLens/pull/749
* Demo colab compatibility by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/752
* Add support for `Mistral-Nemo-Base-2407` model by ryanhoangt in https://github.com/TransformerLensOrg/TransformerLens/pull/751
* Fix the bug that tokenize_and_concatenate function not working for small dataset by xy-z-code in https://github.com/TransformerLensOrg/TransformerLens/pull/725
* added new block for recent diagram, and colab compatibility notebook by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/758
* Add warning and halt execution for incorrect T5 model usage by vatsalrathod16 in https://github.com/TransformerLensOrg/TransformerLens/pull/757
* New issue template for reporting model compatibility by bryce13950 in https://github.com/TransformerLensOrg/TransformerLens/pull/759
* Add configurations for Llama 3.1 models(Llama-3.1-8B and Llama-3.1-70B) by vatsalrathod16 in https://github.com/TransformerLensOrg/TransformerLens/pull/761

New Contributors
* akozlo made their first contribution in https://github.com/TransformerLensOrg/TransformerLens/pull/749
* ryanhoangt made their first contribution in https://github.com/TransformerLensOrg/TransformerLens/pull/751
* xy-z-code made their first contribution in https://github.com/TransformerLensOrg/TransformerLens/pull/725
* vatsalrathod16 made their first contribution in https://github.com/TransformerLensOrg/TransformerLens/pull/757

**Full Changelog**: https://github.com/TransformerLensOrg/TransformerLens/compare/v2.7.1...v2.8.0