Lyman

----------------------

Lyman version 2 comprises a set major change to the library (and an essentially complete rewrite of the codebase). Major aspects of the changes are summarized here; more details are available throughout the collection of related `pull requests <https://github.com/mwaskom/lyman/projects/1>`_.

- The preprocessing workflow now transforms all images into a cross-experiment functional template space that is defined in register with the Freesurfer anatomy. All spatial operations (motion-correction, unwarping, and transformation into template space) are applied in one step to minimize interpolation error.

- Preprocessing has been streamlined to involve mainly spatial transformations of the images, The various signal-processing operations (smoothing, temporal filtering, artifact detection) are now considered part of the modeling workflow, meaning that parameters that control these operations are model-specific. There are no longer separate "smoothed" and "unsmoothed" paths through the workflows.

- Model-fitting and contrast estimation are now implemented in Python, instead of FSL binaries (although univariate model fitting still uses the FILM GLS prewhitening algorithm).

- An experiment-independent ``template`` workflow has been added, and the ``reg`` and ``ffx`` workflows have been removed.

- There is no longer the concept of an "altmodel"; instead, different models are a first-class level of the lyman organization hierarchy.

- Lyman currently supports only HCP-style datasets that include a single pair of spin-echo EPI images with opposite phase encoding directions, which are used for unwarping susceptibility-induced distortions and registration to the anatomy.

- There is currently no support for anatomical normalization or group analyses.

- There is better support for experiments where subjects are scanned in multiple sessions.

- The contrast estimation code now properly handles contrasts between parameter estimates where one of the parameters is undefined in some runs.

- Spatial smoothing is performed using a novel algorithm that smooths in volume space using Gaussian weights determined by distance on the surface manifold.

- A number of static images that were generated for quality control have been removed, and others have been added.

- The command-line interface has changed so that all interaction happens through a single ``lyman`` command line script that has sub-modes corresponding to different workflows (and, in the future, other functionality such as results visualization).

- Automated test coverage of the codebase has been dramatically improved.

- Various aspects of the supporting library code has been moved from `moss <https://github.com/mwaskom/moss>`_ into lyman itself, and moss is no longer a dependency of lyman.

---------------------

This release is being marked as version 1.0 to signify that it is the final iteration of lyman as it existed in the versions leading up to 1.0. From here on, there will be major, breaking changes to the workflows, both from an API perspective and in terms of what kind of analyses are supported. It is possible, but not promised, that minor bugs will be fixed on the 1.0 branch. But going forward, all major development will take place in a 2.0 version that might look substantially different.

- Internal code changes to bring provisional support for Python 3 and later versions of numpy.

- Fixed slice time correction for data collected on a Siemens scanner. The Siemens interleaved pulse sequence changes the order of slices depending on whether there is an odd or even number of slices (for some insane reason). It is now possible to set ``interleaved="siemens"`` in your experiment file to handle this issue properly. See `this blog post <https://practicalfmri.blogspot.com/2012/07/siemens-slice-ordering.html>`_ for more information.

- Made it possible to turn off the FSL highpass filter by setting the ``hpf_cutoff` variable in the experiment file to ``None``.

- The ``lyman.mvpa`` and ``lyman.evoked`` modules have been removed.

- Added the ``view_ffx_results.py`` script, which is a wrapper around ``Freeview`` to boot up a useful visualization of fixed effects statistics on the high-resolution anatomical image and surface mesh.

- Added the ``view_reg.py`` script, which is a wrapper around ``Freeview`` to boot up a useful visualization of the functional-to-anatomical registration quality.

- The top-level cache directory is no longer removed at the end of workflow execution. This means that it is possible to be running multiple workflows (provided that they correspond to different experiments/models) simultaneously without interference from the first one to finish.

----------------------

Command line interface
~~~~~~~~~~~~~~~~~~~~~~

- The full command-line argument namespace is now saved out with the other
experiment parameters for better reproducibility.

Registration workflow
~~~~~~~~~~~~~~~~~~~~~

- The correct registration matrix to go from the epi space to the anatomy is
now written out during the registration so that downstream results from
experiments that use the ``-regexp`` flag are correct.

--------------------------

Preproc workflow
~~~~~~~~~~~~~~~~

- Added the ability to supply fieldmap images that can be used to unwarp
distortions caused by susceptibility regions. This uses FSL's ``topup`` and
``applytopup`` utilities. The images you should provide aren't actually
traditional "fieldmaps", but rather images with normal and reversed phase
encoding directions, from which a map of the distortions can be computed.
See the new experiment option ``fieldmap_template``.


Model workflow
~~~~~~~~~~~~~~

- Added the ability to include additional nuisance variables in the model.
This can now include eigenvariates of deep white matter timeseries and the
mean signal from across the whole brain. This involves the new experiment
options ``wm_components`` and ``confound_sources``. This change requires that
you rerun the preproc workflow before running the model workflow after
updating lyman.

- Made the inclusion of artifact indicator vectors in the design matrix
optional. See the new experiment option ``remove_artifacts``.

- Made some changes to the model summary node to use memory more efficiently.
The model summary code should now use a similar amount of memory as the
``film_gls`` model fitting process.


Fixed effects workflow
~~~~~~~~~~~~~~~~~~~~~~

- Fixed effects workflow now saves out a mean functional image that is the
grand mean across runs.

----------------------

Model workflow
~~~~~~~~~~~~~~

- Upgraded the model workflow to be compatible with FSL 5.0.7 and later. If you
upgrade lyman, you will have to upgrade your FSL installation (i.e. it does
not maintain backwards compatibility with older FSL). You should also upgrade
to nipype 0.10. The main advantage of upgrading should be increased memory
performance in the model estimation.

Registration workflow
~~~~~~~~~~~~~~~~~~~~~

- Added the ability to do cross-experiment registration, e.g. in the case where
you have a functional localizer. This is accomplished through the ``-regexp``
flag in ``run_fmri.py``. For example, the cmdline ``run_fmri.py -exp A
-regexp B -regspace epi -timeseries`` will combine the func-to-anat matrices
from experiment A and the anat-to-func matrix from the first run of
experiment B, placing the experiment A files in a common space with
experiment B files.

Fixed-effects workflow
~~~~~~~~~~~~~~~~~~~~~~

- The fixed effects analysis no longer crashes when a subject did not have any
observations for an event.

Mixed-effects workflow
~~~~~~~~~~~~~~~~~~~~~~

- The mixed effects workflow now excludes empty images, allowing you to run it
on a group where some subjects did not have any observations for a specific
event/contrast.
- Updated the mixed effects boxplot code for compatibility with seaborn 0.6.

Anatomy snapshots script
~~~~~~~~~~~~~~~~~~~~~~~~

- Added plots of the native white and pial surfaces
- Surface plots are now saved in one image file with all views, and the subplot
size is automatically inferred to maximize the usage of space
- Added ventral views to the surface images
- Changed how the surface normalization is summarized. The new visualization
highlights vertices where the binarized curvature value is different between
the normalized subject and template
- Remove the "-noclose" option, as better ways to avoid the problem that
motivated it have been identified.

Note that there are corresponding changes in ziegler that are needed to
properly view the new images, and there isn't backwards compatibility
with the old outputs. This script can be rerun on older lyman analyses
without affecting any results.

Surface snapshots script
~~~~~~~~~~~~~~~~~~~~~~~~

- Changed how the individual frames of the surface snapshots are stitched
together to maximize the use of space. This (and the changes in the anatomy
snapshots script) rely on some new functions in ``lyman.tools.plotting``
that may be generally useful.
- Remove the "-keep-open" option, as better ways to avoid the problem that
motivated it have been identified, and removed the "-no-window" option,
as it is not clear whether this ever worked.

--------------------------

Execution
~~~~~~~~~

- Added the option to submit jobs using slurm.
- Added the ``crash_dir`` parameter at the project level (i.e. it will be
defined when you run ``setup_project.py`` and will be stored in
``$LYMAN_DIR/project.py``). This allows you to specify where debugging
information will be written if something goes wrong during workflow
execution. The previous approach to selecting where crash files would be
written was not robust in all execution contexts. **Important:** if you
upgrade to this version and try to rerun something in an existing project,
you will get an error. This can be avoided by defining ``crash_dir`` in your
project file. These files are usually only transiently useful, so the default
location for new projects is ``/tmp/nipype-$USER-crashes``.

Registration workflow
~~~~~~~~~~~~~~~~~~~~~~

- Added ability to register the residual timeseries after model-fitting, using
the ``-residual`` flag in ``run_fmri.py`` when ``reg`` is in the workflow
spec. This file will be called ``res4d_xfm.nii.gz`` in the registration
output.