Overhaul of statistical analysis pipeline
We refactored the statistical analysis pipeline so that it no longer depends on jupyter notebook extensions (many users reported issues installing the extensions). **Note: if you have already run the analysis pipeline, your saved group labels and syllable names will no longer be usable after this version update unless you run the conversion code below.**
Analysis pipeline conversion code
Previously, group labels and syllable names were stored in yaml files (`index.yaml` and `syll_info.yaml` respectively). As of this update, they will be stored in csv files. To convert from yaml to csv, use the conversion code below:
project_dir = XXX
model_name = XXX
import os
import yaml
import pandas as pd
convert index.yaml
index_filepath = os.path.join(project_dir, 'index.yaml')
with open(index_filepath, 'r') as f:
index_data = yaml.safe_load(f)
index_df = pd.DataFrame({"group": [i["group"] for i in index_data["files"]],"name": [i["name"] for i in index_data["files"]]})
write index dataframe to csv
index_df[["name","group"]].to_csv(os.path.join(project_dir, "index.csv"), index=False)
convert syll_info.yaml
syll_info_path = os.path.join(project_dir, model_name, "syll_info.yaml")
with open(syll_info_path, "r") as f:
syll_info = yaml.safe_load(f)
syll_info_df = {"syllable": [], "short_description": [], "label": [], "movie_path": []}
for k, v in syll_info.items():
syll_info_df["syllable"].append(k)
syll_info_df["short_description"].append(v["desc"])
syll_info_df["label"].append(v["label"])
syll_info_df["movie_path"].append(v["movie_path"])
syll_info_df = pd.DataFrame(syll_info_df)
syll_info_df.to_csv(os.path.join(project_dir, model_name, "syll_info.csv"), index=False)
Other main changes
Support for 3D grid movies
It is now possible to generate grid movies based on multi-camera 3D keypoint tracking. See [docs](https://keypoint-moseq.readthedocs.io/en/latest/FAQs.html#making-grid-movies-for-3d-data) for details.
Automatic kappa scans
We wrote example code for performing a sequence model fits with different values of the stickiness hyperparameter (kappa), and also added a new function to help visualize the results. See [docs](https://keypoint-moseq.readthedocs.io/en/latest/advanced.html#automatic-kappa-scan) for details.
Calibration bug-fix
Many users were reporting bugs in the calibration step of the keypoint-MoSeq pipeline, which were caused by recent version releases in the holoviews/panel/jupyter ecosystem. We updated the code to fix these bugs. If you still encounter bugs with the calibration widget, try upgrading the relevant packages:
pip install holoviews --upgrade
pip install bokeh --upgrade
pip install panel --upgrade
pip install ipywidgets --upgrade
pip install jupyter --upgrade
**Note: From now on, the calibration widget will only work in jupyter lab. It will not work in jupyter notebook.**
Other main changes
New FAQs section on visualization
We added a [new FAQ section](https://keypoint-moseq.readthedocs.io/en/latest/FAQs.html#visualization) about visualization. The section covers:
- Making grid movies for 3D data
- Why are there only trajectory plots for a subset of syllables?
- Why are there only grid movies for a subset of syllables?
- Why do my trajectory plots and grid movies disagree?
New FAQ section on "Advanced topics" in modeling
We added a [new FAQ section](https://keypoint-moseq.readthedocs.io/en/latest/advanced.html#) about advanced modeling topics. The section covers:
- Exporting model-based pose estimates
- Automatic kappa scans
Removed code for model-free changepoint analysis.
The original keypoint-MoSeq reprint included an analysis of changepoints and a complicated procedure to identifying them. These have mostly been removed in the revision, so we are deleting them here too.
Other minor changes
- Return sampled instances in generate_grid_movies
- Allow passing arbitrary keypoint colors in viz functions
- Allow manual setting of axis lims for trajectory plots
- Allow setting "jitter" and "seed" inputs in model fitting calls
- Make "density_sample" a separae kwarg in viz functions
- Small changes to wording of docs
- Bugfixes
New Contributors
* NeuTTH made their first contribution in https://github.com/dattalab/keypoint-moseq/pull/106
**Full Changelog**: https://github.com/dattalab/keypoint-moseq/compare/0.2.5...0.3.0