Pysipfenn

Minor Changes:
- The `OPTIMADEAdjuster` model tuning class now includes more explicit provenance tracking through its `references` attribute, populated if the provider response includes "relations" to the references. As of now, many popular providers do not include these, but some, like `alexandria`, can be used and should populate `references` with a list of lists of DOI strings.
- The `KS2022_randomSolutions`'s optional metadata dictionary, returned when `returnMeta=True` is passed to `generate_descriptor` function, now includes a list of KS2022 featurizations of each supercell expansion (i.e., iteration) under the `individualResults` field, which can be used in addition to the converged global ensemble. Among several use cases, users can use these to run ML models over subsets of the global ensemble and look at the prediction distributions to gain additional insights. E.g.:
<img width="1043" alt="KS2022_RSS_Distributions" src="https://github.com/user-attachments/assets/7d91f15f-3dd0-4b5f-8156-b6e175173ec3">
- `KS2022_randomSolutions` implementation and code style have been generally improved.
- Appropriate documentation and tests were added.

**Full Changelog**: https://github.com/PhasesResearchLab/pySIPFENN/compare/v0.16.1...v0.16.2

Minor Changes:
- Mitigate `pymatgen`'s incompatibility (currently preventing installation) with `numpy>=2` by setting the maximum version
- Added a convenience function in misc for patching pymatgen's `core/periodic_table.json` for exotic elements like Livermorium of Hessium.
- Added a simple cubic (`SC`) prototype of Po to the library.
- Other minor updates.

**Full Changelog**: https://github.com/PhasesResearchLab/pySIPFENN/compare/v0.16.0...v0.16.1

**This version introduces 3 exciting changes!** (1) The all new [`ModelAdjusters`](https://github.com/PhasesResearchLab/pySIPFENN/blob/main/pysipfenn/core/modelAdjusters.py) submodule automates tuning and can fetch data directly from [`OPTIMADE API`](https://www.optimade.org); (2) A new manuscript detailing advantages of our featurization tools has been put on [arXiv:2404.02849](https://arxiv.org/abs/2404.02849); and (3) the name of the software was updated to **py**thon toolset for **S**tructure-**I**nformed **P**roperty and **F**eature **E**ngineering with **N**eural **N**etworks to retain the `pySIPFENN` acronym but better reflect our strengths and development direction.

Major Changes:
- Submodule of [`ModelAdjusters`](https://github.com/PhasesResearchLab/pySIPFENN/blob/main/pysipfenn/core/modelAdjusters.py) has been set up for all kinds of model adjusting efforts.
- The [`LocalAdjuster`](https://github.com/PhasesResearchLab/pySIPFENN/blob/5f20178937f1a5f2b1af309a1e87a843e089c3f5/pysipfenn/core/modelAdjusters.py#L22) implements local model tuning, plotting of results, and hyperparameter matrix search.
- The [`OPTIMADEAdjuster`](https://github.com/PhasesResearchLab/pySIPFENN/blob/5f20178937f1a5f2b1af309a1e87a843e089c3f5/pysipfenn/core/modelAdjusters.py#L637) class combines it with the powerful [`OPTIMADE API`](https://www.optimade.org) to automate data fetching. Now, you can quickly
python
from pysipfenn import Calculator, OPTIMADEAdjuster
c = Calculator(autoLoad=False)
c.loadModels("SIPFENN_Krajewski2022_NN30")
ma = OPTIMADEAdjuster(c,
model="SIPFENN_Krajewski2022_NN30",
provider="mp",
targetPath=("attributes", "_mp_stability", "gga_gga+u", "formation_energy_per_atom"),
device="mps" MPS is for Apple M-series GPU
)

ma.fetchAndFeturize(
'elements HAS "Hf" AND elements HAS "Mo" AND NOT elements HAS ANY "O","C","F","Cl","S"',
parallelWorkers=4)
ma.adjust()

ma.plotStarting() See the starting performance
ma.plotAdjusted() See the adjusted performance

or to perform a hyperparameter search, replace the `ma.adjust()` with:
python
ma.matrixHyperParameterSearch()
ma.adjust(learningRate=0.0001, optimizer='AdamW', weightDecay=1e-05, epochs=37)

- The new manuscript on _Efficient Structure-Informed Featurization and Property Prediction of Ordered, Dilute, and Random Atomic Structures_ has been uploaded to [arXiv:2404.02849](https://arxiv.org/abs/2404.02849) and will be submitted to journal in a couple days after comments from collaborators.


Minor Changes:
- Added `writeDescriptorsToNPY` function to streamline persisting feature data into NumPy for our end-users. Appropriate tests were added. Thanks rdamaral for making this contribution!
- Improved numerous docstrings.
- Random solution featurizer now exits gently on `KeyboardInterrupt`
- The `LocalAdjuster` and `OPTIMADEAdjuster` were (optionally) connected to [ClearML](https://clear.ml) for neat tracking of ML training histories.
- Minor bugfixes in several spots.


New Contributors
* rdamaral made their first contribution in https://github.com/PhasesResearchLab/pySIPFENN/pull/15

**Full Changelog**: https://github.com/PhasesResearchLab/pySIPFENN/compare/v0.15.1...v0.16.0

Major Changes:
- The printout of `Calculator` initialization, model parsing, prototype parsing, etc., is now colorized (granted `verbose=True`) for quick and effortless reading of critical settings and information.
- Model download is now handled by our in-house-maintained `pysmartdl2`, instead of `pySmartDL`. It allows us to better test CI with modern Python versions and ensure long term stability.

Minor Changes:
- Requires `pymatgen` version `2024.2.20`, which fixed a serious security issue in `pymatgen.io`.
- Small bug fixes for two rare cases.
- Workshop materials in `examples` updated to be compatible with the current `pymatgen` version.
- Minor style improvements.


**Full Changelog**: https://github.com/PhasesResearchLab/pySIPFENN/compare/v0.15.0...v0.15.1

Welcome to pySIPFENN v0.15, which is ***our biggest singular jump since 2020 spanning 196 commits since v0.13.0*** 🎉 It covers improvements in nearly every place of the codebase, including ***new features, better performance and more convenient high-level API***. Two key new features are (1) a ***neat prototype library*** (v0.14) simplifying the handling of atomic structures, and (2) ***random solid solution (RSS) featurizer*** / descriptor calculator (v0.15), which allows you to featurize chemical composition occupying a prototype structure.

Now, you can efficiently utilize pySIPFENN to, ***for instance, featurize high entropy alloys (HEAs) occupying BCC, FCC, C14, and C15 structures.*** Have a look below for details!

Major Changes:

Random Solid Solution Featurization
- KS2022 Random Solution Featurization ([`KS2022_randomSolutions`](pysipfenn/descriptorDefinitions/KS2022_randomSolutions.py)) added by amkrajewski (mostly in https://github.com/PhasesResearchLab/pySIPFENN/pull/10), which **allows users to quickly obtain KS2022 feature vector corresponding to a random solid solution with composition `comp` occupying arbitrary structure `struct` (including library `BCC`, `FCC`, `HCP`) through iterative expansion of ensemble of local chemical environments (LCEs) until all features, i.e., statistics over LCEs, and composition converge onto stable values.**
- This method is philosophically similar to the SQS approach [(10.1103/PhysRevLett.65.353)](https://doi.org/10.1103/PhysRevLett.65.353), which one can generate with [Alloy Theoretic Automated Toolkit](https://www.brown.edu/Departments/Engineering/Labs/avdw/atat/) (ATAT) [(10.1016/j.calphad.2013.06.006)](https://doi.org/10.1016/j.calphad.2013.06.006) or approximate with [sqsgenerator](https://github.com/dgehringer/sqsgenerator); however, here, we are not only encoding the correlations between anonymous species types but also (1) consider differences between them, thus, e.g., solutions of elements with similar electronic configurations will converge faster) and (2) allow arbitrary complexity of the chemical composition under a single methodology.
- It should also be better at capturing randomly occurring uncommon extreme-case events, which may not occur in low-correlation cases.
- We have tuned the default values of the method for complex cases of (a) high entropy alloys (HEAs) with 6+ components and (b) multicomponent steels with several minor alloying elements.
- Added matching tests for all methods of [`KS2022_randomSolutions`](pysipfenn/descriptorDefinitions/KS2022_randomSolutions.py) featurizer. Added documentation for all tests, so that they can serve as runnable examples.
- Added complete API documentation of all methods and all parameters, with both meaning and motivation, for [`KS2022_randomSolutions`](pysipfenn/descriptorDefinitions/KS2022_randomSolutions.py).
- Added top-level [`Calculator.calculate_KS2022_randomSolutions()`](pysipfenn/core/pysipfenn.pyL480) and [`Calculator.runModels_randomSolutions()`](pysipfenn/core/pysipfenn.pyL800) for conveniently calling everything on complex problems, even with mixed-type inputs like:
python
c.calculate_KS2022_randomSolutions(
['FCC', myDistortedBCC, 'BCC', 'HCP'],
['WMo', Composition.from_weight_dict({'Mo': 20, 'W': 70, 'Zr': 10}), 'FeNi', 'CrNi'],
mode='parallel',
max_workers=4)




Prototype Library:
- The new prototype library functionality allows you to store commonly used structures in a consistent fashion and pleasant schema under [`misc/prototypeLibrary.yaml`](pysipfenn/misc/prototypeLibrary.yaml) file in your pySIPFENN installation. It is initialized alongside every `Calculator` instance.
- Parsing custom libraries is very easy and can be done from a file in your directory or from a remote repository like Zenodo! Simply use the [`parsePrototypeLibrary`](pysipfenn/core/pysipfenn.pyL130) to fetch it. Or, to append your library with an external one and persist it your installation, use [`appendPrototypeLibrary`](pysipfenn/core/pysipfenn.pyL180).
yaml
- name: BCC
origin: https://www.oqmd.org/materials/prototype/A2_W
POSCAR: |
W
1.0
-1.58250 1.58250 1.58250
1.58250 -1.58250 1.58250
1.58250 1.58250 -1.58250
W
1
Direct
0.00000 0.00000 0.00000

- Added complete API documentation of all methods and all parameters, with both meaning and motivation, for everything prototypeLibrary-related.
- Added matching tests for all new prototypeLibrary-related methods. Added documentation for all tests, so that they can serve as runnable examples.

General:
- Complete overhaul of the documentation in terms of consistent styling, type-hinting, and extent of the content. Most functions can be well-understood just by looking at information displayed by IDE like PyCharm or VSCode.
- The `Calculator` class now can self-destruct and deallocate by call to its `destroy()` function, allowing users to better manage memory used in calculations on low-power systems.
- Performance improvements in chemical attributes calculation of all `KS2022`-calculating featurizers.
- Performance improvements and minor fixes in `KS2022_dilute` featurizer.
- Performance and feature improvements in the [`modelExporters`](pysipfenn/core/modelExporters.py), including imports optimization so that are now they only run when needed by given exporter class.
- Optimized testing workflows in terms of events triggering them and general performance.
- Improved testing clarity and platform compatibility, by splitting them into individual workflows.
[](https://github.com/PhasesResearchLab/pySIPFENN/actions/workflows/coreTests_LinuxUbuntu.yaml) [](https://github.com/PhasesResearchLab/pySIPFENN/actions/workflows/coreTests_MacM1.yaml) [](https://github.com/PhasesResearchLab/pySIPFENN/actions/workflows/coreTests_MacIntel.yaml) [](https://github.com/PhasesResearchLab/pySIPFENN/actions/workflows/coreTests_Windows.yaml)
- Dependencies were analyzed for compatibility and set in a way to ensure long-term maintainability and near-future compatibility with Python 3.12.
- Added [instructions for contributing to pySIPFENN](https://pysipfenn.readthedocs.io/en/latest/contributing.html), explaining philosophy of the software and several design choices.

Minor Changes:
- Improved printouts from `Calculator` initialization and current state reporting.
- Improved verbosity handling in `Calculator` instances.
- `CITATION.cff` added for clear citation guidance in the future.
- Documentation now includes [miscellaneous notes](https://pysipfenn.readthedocs.io/en/latest/miscellaneousnotes.html) users may find useful.
- `.gitignore` improvements as requested by users.
- Added testing for several smaller functions like `ward2ks2022`, to increase coverage by 1%. Now it is over 95%.
- Code style improvements.
- Commenting improvements scattered throughout the codebase to make modifying our code easier.
- And many more added on the way :)

Notices:
- In the near future (likely next release), pySIPFENN will drop **official support for Python 3.9**, due to (1) noticeably lower performance, (2) deprecation of 3.9 on some platforms (MacOS14), and (3) next release using several pure-Python features added in Python 3.9.
- In the near future (likely next release), pySIPFENN will **add support for Python 3.12** to keep it up-to-date and take advantage of performance boost. Our codebase is ready for the switch. We are waiting for PyTorch support for it in its stable release.

**Full Changelog**: https://github.com/PhasesResearchLab/pySIPFENN/compare/v0.13.1...v0.15.0

Major Changes
- All tests are now documented under API [pySIPFENN Tests](https://pysipfenn.readthedocs.io/en/latest/source/pysipfenn.tests.html#pysipfenn-tests) page with hyperlinks to their sources, acting as minimalistic examples of using the software.

Minor Changes
- Improvements to FAQ, docs index, and README.
- Polishing of the documentation page and general maintenance.

**Full Changelog**: https://github.com/PhasesResearchLab/pySIPFENN/compare/v0.13.0...v0.13.1