Python-catmaid

Contributors: Albert Cardona, Andrew Champion, Tom Kazimiers


Features and enhancements

Connectivity widget:

- Partners can now be filtered by a minimum threshold on number of nodes, rather
than only being able to filter single-node partners.

- The user's review team is now an option in the partner review filter.

- Changing the review filter no longer reloads the entire widget.

- Many small performance improvements.


Selection table:

- The table layout has been streamlined with other tables in CATMAID. All
columns except 'action columns' can now be used for sorting. Pagination is now
done with the buttons to the right above and below the table, the page length
can now be adjusted, too. The summary info button moved into the button panel
while the filter input is now part of the table.

- It is now possible to add a new annotation to individual neurons without
changing the current selection. This can be done with the little tag icon in
the actions column on the right. The former info button was replaced by a
small 'i' icon and clicking the folder icon in the same column will open a
Neuron Navigator window for the respective neuron.

- All visibility related colums can be hidden with a new checkbox in the button
panel. This might be useful to save space if a selection table is not used to
control a 3D viewer.


Miscellaneous:

- Neuron search: annotations can now be searched for those by users in the
review team.

- Log: entries can now be filtered to include only actions from the user's
review team.

- The maximum number of nodes returned to the tracing overlay is now
configurable as a server setting: NODE_LIST_MAXIMUM_COUNT (default 5000).

- Group graph: a new lock buttons in the Selection tab allows to lock selected
nodes, so that their position doesn't change until they are unlocked again.


Bug fixes

- Fix display of intermediate nodes on edge between two joined skeletons.

- The last lines of the review widget were hidden sometimes. This is not the
case anymore.

CATMAID now uses the GPLv3 license and moved away from the stricter
AGPLv3. This move was discussed with all previous contributors and
agreed on. See the corresponding commit for more details.

Contributors: Albert Cardona, Andrew Champion, Tom Kazimiers


Notes

This release includes database changes that require manual intervention if you
are upgrading from an existing installation. A new dependency is now required:
PostGIS, an extension to PostgreSQL. After its installation, it has to be
activated for the CATMAID database. To do so, connect to the database using a
Postgres system user. Assuming a Postgres system user named "postgres" and a
CATMAID database named "catmaid", this could be done by calling

sudo -u postgres psql -d catmaid

Being connected to the database, PostGIS can be enabled by executing

CREATE EXTENSION postgis;

Now PostGIS is enabled and the connection can be closed again. Now a regular
update can be performed. Please note that this update can take quite some time
to complete. On bigger neuron tracing installations, multiple hours are
realistic.


Features and enhancements

Key shortcuts / mouse operations:

- Ctrl + mouse wheel now zooms the stack, while shift zooms by smaller
increments.

- Pressing X in the tracing tool will begin measuring distance from the current
cursor position. The mouse wheel can be used to measure along the stack Z
axis. Clicking will close the measurement tool and show the final distance in
the status bar. ESC cancels the tool.


Multi-view tracing and virtual nodes:

- Orthogonal views on a regular XY stack can now also be used for neuron
reconstruction. If they are available as CATMAID stacks and opened while the
tracing tool is activated, tracing data will be shown in the respective
orthogonal views as well. Tracing can be done in these views just like in the
regular XY view.

- When tracing, it is not required anymore to place a node in every section. If
no node has been placed in a section, CATMAID will place a so called virtual node
where the skeleton and the section meet. If this virtual node is modified in
any way, e.g. tagging, joining, moving, etc. it will be created. This also
slightly changes the way reviews work. Review information is only stored on
real nodes.

- The review widget has a new settings: in-between node step. It specifies how
many sections can be skipped between adjacent real nodes. This is done with
respect to the currently focused stack. This stack is also used to determine
in which direction to move to look beyond the start of a segment.


Stack viewer:

- Other stacks in a project can be added to an open stack view by selecting the
"Add to focused viewer" option from the stacks menu. This allows multiple
stacks to exist in the same view like overlays, while accounting for
differences in translation and resolution. The navigator will expand the
available zoom levels to accomodate the maximum and minimum zoom possible in
all of the open stacks.

- Tile layers for stacks added to a viewer can be removed from a viewer via an
"x" in the tile layer control.

- Multiple viewers into the same stack can now be opened.

- Each stack viewer can be toggled between coupling its navigation with other
open stack viewers. Toggle this via the "Navigate with project" checkbox in
the tile layer control.


Tile layer:

- New filter (WebGL only): intensity thresholded transparency


3D Viewer:

- Connector restriction can now explicitly be turned on and off with a pull
down list. One can select between "Show all" (i.e. restriction turned off),
"All shared connectors" will only show connectors with partners in the current
selection and "All pre->post connectors" will only allow connectors with at
least one presynaptic partner and one postsynaptic partner in the current
selection. The last option, "All group shared" allows to select two skeleton
sources (e.g. two selection table) and it will only show connectors that are
part of the 3D viewer and that connect between both selected groups of
skeletons. There is also a pre->post enforcing variant of it.


Neuron search:

- The name displayed for neurons now follows the same naming mechanism used for
other widgets. It can be controlled through the settings widget and will
automatically update if a neuron name is changed.


Miscellaneous:

- In the connectivity widget, upstream and downstream thresholds can now be set
at once for all seed neurons. Two drop down controls used for this will be
displayed if there is more than one seed neuron.

- Treenode Table refurbishing: far faster, supports multiple skeletons, can do
tag search with regular expressions and lists the skeleton treenode ID.

- Rows and columns of the connectivity matrix can now be moved around with
little buttons that appear when the mouse is over row and column header cells.

- There are now three options to change focus if a pointer enters a window:
don't change focus (how it has been so far), focus stacks (will activate
stacks when hovered, but won't change focus for other windows) and focus all
(will change focus to every window hovered). The default will be stack focus
follows the pointer. The settings widget makes these options available in
general settings area.

- There are three new controls in the split/merge dialog: Toggle the
display of input and output markers on neurons in the embedded 3D viewer and
select which shading method is used (default: "active node split").
Alternatively, "Strahler index" coloring can be used, which helps with
depth perception.

- Attempting to reload a CATMAID browser tab or go back in history, will now
result in a warning dialog, asking for confirmation. It makes clear that
CATMAID's window layout and content won't be saved if the acton isn't
canceled.


Administration:

- Now that virtual nodes are available, existing database can (but don't have
to) be optimized. A new management command will look for straight skeleton
parts that are not referenced in any way and prunes them. In other words, if
there are three successive collinear nodes and the middle one is not
referenced, it will be removed.

manage.py catmaid_prune_skeletons


Bug fixes

- The Neuron Search widget doesn't throw an error anymore when a neuron listed
in it is merged.


- There is no longer a race condition in the database during concurrent
split/merge of a skeleton and creation of a treenode in that skeleton. While
this is not a comprehensive guarantee of conflict-free concurrency, it does
remove the most likely scenario resulting in corruption of the database model.

Bug fixes

- Fix radius based neuron selection in tracing window.

Contributors: Tom Kazimiers


Features and enhancements

Key shortcuts / mouse operations:

- Cycling through open end nodes will now only visit the root node if it is an
actual leaf. That is, when it has only one child node and is untagged.


3D Viewer:

- A light background shading variant for connectors was added. It uses a darker
cyan color which provides more contrast if a white background is used.


Miscellaneous:

- The location of messages and notifications can be configured in the settings
widget. The default location is still the upper right corner.

- If the node display limit is hit while panning the field of view in tracing
mode, node refresh will be temporary disabled. Once the mouse button is
released again an no further panning happens within one second, node update is
reset to normal. This allows for smoother panning if many nodes are visible.


Bug fixes

Review system:

- Review teams are now respected when Shift + W is used to jump to the next
unreviewed node.


3D viewer:

- Skeletons with other coloring than "Source", will now be visible when exported
as SVG in the 3D viewer.


Miscellaneous:

- Skeletons added to a selection table, will now honor the table's "global"
settings for pre, post, meta and text visibility.

- If an annotation is removed from a neuron, the annotation itself will be
deleted, too, if it is not used anywhere else. Now also meta annotations of
the deleted annotation will be removed (and their meta annotations...), if
they are not used anywhere else.

Contributors: Albert Cardona, Andrew Champion, Tom Kazimiers

Features and enhancements

Connectivity widget:

- Partner filtering now supports regular expressions when the first character
of the search input is "/".


Treenode table:

- *REMOVED*: Radii can no longer be edited by clicking on their cell in the
table.


Connectivity matrix:

- The tracing tool has got a new widget: a connectivity matrix. It can be opened
with the "M" frame icon next to the button for the connectivity widget. To use
it, one has to append skeletons for its rows and columns. Skeletons can also
be added as group. Each cell shows two sub-cells, the first one shows the
number of synapses from row to column and the second one the number synapses
from column to row. When a synapse count number is clicked, a connector
selection is opened, that contains the corresponding synapses. Both pre- and
post-synaptic count cells can be colored individually. By default a coloring
similar to the tracing layer's red and cyan is used. There are also color
gradients available to produce heat maps (i.e. color cells based on the
actual synapse count).

- Graph widget: ability to split neurons by text tag on their skeletons. It's the
"Tag" button under the "Subgraph" tab. Enables you to manually define regions
on a neuronal arbor (like axon and dendrite, or multiple dendritic domains)
and then have them be represented each as a node in the graph. The skeleton
will be virtually cut at the nodes containing the tags, with the tagged node
belonging to the downstream part (relative to the root node).


3D Viewer:

- Color mode "Downstream of tag" is now a shading mode.

- New synapse coloring mode "Same as skeleton". If you then hide the
skeletons and show only the synapses you will e.g. see spatial tiling of ORN
axons, each defining a glomerulus in the larval olfactory lobe.

- For PNG and SVG export one can now specify the dimensions of the result files.
A dialog shown before exporting asks for width and height.


Neuron dendrogram:

- The horizontal and vertical spacing between nodes in the neuron dendrogram can
now be fine tuned.


Administration:

- A new tool 'Group membership helper' has been added to add multiple users to
multiple groups or to revoke their group membership. This can be used to
control access over the data created by individual users.


Miscellaneous:

- A node-placement-and-radius-edit mode has been added. If enabled through the
settings widget (Tracing > "Edit radius after node creation"), the radius for
a node will be edited immediately after it has been created. This allows for
easier volumetric reconstruction. In this mode, the radius circle editing tool
is used to specify the radius. No dialog is shown once a radius is selected for
a node and it will only be saved for the new node.

- A new connector type ("abutting") can now be created. In contrast to the
regular synaptic connector, it can be used to represent the fact that two or
more neurons are in abutting processes. For now this mode can be activated
through the settings widget (Tracing > "Create abutting connectors"). For
abutting connectors the lines representing the links to nodes will appear in a
green color.


Bug fixes

3D viewer:

- Adding and removing neurons and static data lead in some situations to many
errors that were displayed on the console (and therefore not visible to most
users) and caused minor performance problems. This has been fixed and all data
should now be added and removed correctly.

- Following the active node should now work much more reliable. Before, it could
happen that this stopped working after a second 3D viewer was closed.


Connectivity widget:

- Fix one cause of sluggish behavior for widgets that have been modified many
times. Also fixes repeated alert dialogs when clicking a neuron in the
partner tables that no longer exists or does not have any treenodes.


Neuron Navigator:

- Don't show an error if an invalid regular expression was entered for
searching. Instead, color the search box red and show a warning message.

Contributors: Albert Cardona, Andrew Champion, Tom Kazimiers, Stephan Gerhard

Features and enhancements

Key shortcuts / mouse operations:

- Shift+T removes all tags from the currently active node.

- After using R to go the nearest open leaf, shift+R cycles through other open
leaves in the skeleton in order of ascending distance from the starting
location. Combining alt with these operations orders open leaves by most
recent creation instead of distance.

- Ctrl+Y removes the active skeleton from the last used selection widget.

- Shift+Y selects skeletons within a radius of the active node in the tracing
layer and adds them to the last used selection widget. Ctrl+shift+Y works in
the same way to remove skeletons from the last used selection widget.

- If the next (or previous) branch/end point is already selected when V (or B)
is pressed, the view will center on it nevertheless.

- If the mouse is over the stack when zooming, the view will recenter so that
the same stack location remains under the mouse at the new scale.

- During review, Q and W during will refocus on the last reviewed neuron if
review is interrupted (another node is selected), regardless of the auto
centering setting. If one looks beyond the current segment, the last reviewed
node will be selected by Q and W as well, but auto centering is respected.


Review system:

- New "Reviewer Team" system allows filtering reviews in visualizations and
statistics to include only those by particular reviewers. Each user can
control which reviewers to include in her team. A date can be configured for
each reviewer in the team, so that only reviews from that reviewer after this
date are included.
* A user's reviewer team is configured through the Settings widget.
* The review widget includes a team column between the user and union columns.
* The percent reviewed column in the selection widget can be set to team or
union.
* Team review coloring is available in the 3D viewer and group graph.


3D viewer:

- With Ctrl + mouse wheel, only the camera is moved in target direction, the
target stays fixed. If Alt + mouse wheel is used, the target moves as well.

- The CSV export not also includes the parent ID for each node, which can be
used to reconstruct the topological tree.

- The auto-created selection widget is now 50% smaller, giving more vertical
space to the 3D viewer.

- With the help of controls of the Animation tab, simple animations can be
played. Currently, rotation around the X, Y and Z axis as well as the current
"up" direction of the camera. is supported. The back-and-forth mode will
reverse rotation direction once a full circle is reached. With the help of the
stepwise visibility option, individual neurons can be made visible after a
certain amount of time the animation is running. Additionally, neurons can be
made sequentially visible after each rotation.

- Animations can also be exported as WebM movie file. The "Export animation"
button in the Export tab, will show a dialog with basic export settings. Like
with the other view export options, the current 3D view setup is used. The
frame size can be adjusted in the export dialog. Creating the file can take
some seconds and currently only works for the Chrome browser (due to the lack
of WebP support in others). The resulting WebM video file can be converted to
any other format using e.g. VLC player, if needed.

- New shading mode "synapse-free chunks". Has one parameter, the minimum amount
of synapse-free cable to consider between two consecutive synapses, adjustable
from the "Shading Parameters" tab.

- New shading mode "dendritic backbone". Depends on 'microtubules end' tags, or
will approximate twigs by using the Strahler number entered in the "Shading
Parameters" tab.

- The view settings tab now contains a control to scale the size of the node
handles (e.g. active node, special tags).


Tile layer:

- Tiles can now be rendered with WebGL, which enables new visualization features
and fixes some flickering issues. Enable via "Prefer WebGL Layers" in
Settings. The WebGL renderer is currently considered experimental and may have
stability issues on some clients. See
https://github.com/catmaid/CATMAID/issues/186#issuecomment-86540706 for
details on using WebGL layers with your image stack host.

- The blend mode used to combine stacks and overlays is now configurable when
using WebGL. This greatly improves visualization of confocal and other
multichannel data. Blend mode is selectable from the layers control, activated
via the toggle at the bottom left of the stack view.

- Filters can be applied to layers when using WebGL. Filters can be added and
removed from layers through the layers control. Available filters currently
include:
* Gaussian blur
* Color inversion
* Brightness, contrast and saturation adjustment
* Color matrix transform


Connectivity widget:

- It is now possible to remove added neurons again. Each row of the table of
target neurons now contains a small 'x' icon in the first column to remove it.

- The selection column is not included anymore in the CSV export.


Analyze Arbor:

- Options are provided to approximate twigs by using a branch Strahler number
defined in the "Options".

- Dimensions of the pie charts and XY plots is now configurable from the
"Options" dialog.


Graph widget:

- New button to "Clone" the graph widget: opens a new widget with identical content.

- New buttons to "Save" and "Open..." to/from JSON, so that complex graphs can be
reloaded later on. Skeletons not present in the database are not loaded.


Miscellaneous:

- Selecting tags for highlighting in the neuron dendrogram

- Synchronization between widgets was improved. Deleting a neuron in one widget,
will remove it from other widgets as well.

- Hovering over the CATMAID text on the front page will display CATMAID's
version.


Admin:

- For projects, stacks, overlays and data views there is now the option to
duplicate objects from within the admin view. To copy objects without their
relations, there is now a new action in the list view's action menu. To
duplicate an entity with its relations, select the object and use the "save as
new" button.


Export:

- A basic JSON export of all treenodes and connectors of the selected neurons is
now possible.


Bug fixes

Tracing overlay:

- Trying to remove a non-existent tag from a node now doesn't show an error
dialog anymore, but only a notification.


Key shortcuts / mouse operations:

- Fix bug where tagged nodes were not considered open by R regardless of tag
content.


Neuron search:

- Make neuron names wrap and use the next line, if there is not enough space for
it. This makes the table not expand in width until the name fits anymore.


3D viewer:

- Picking a synapse or other selectable elements is now more robust and now
works also in orthographic mode.

- The projection mode (orthographic or perspective) is now also stored in a
saved view.

- The 3D viewer's drawing canvas is now correctly sized again. Since the tab
panel has been introduced, the 3D viewer has been too high. Now the
pre-defined views (XY, XZ, ZY, ZX) are display correctly again, i.e. the whole
bounding box is now seen again.

- Performance enhancement when smoothing skeletons with a Gaussian by avoiding
to update the same Vector3 instances twice.


Reviews:

- Pressing 'E' during review will now go to the next unreviewed segment as seen
from the currently reviewed one. Before, the first unreviewed segment as seen
from the top of the table was selected.

- Pressing 'Q' on the first node (leaf) brings one back one section to check if
the segment really ends. Pressing 'W' afterwards now brings one back to the
first node, not the second like it has been before.


Connectivity widget:

- CSV export works again.


Miscellaneous:

- Vertical resizing of widgets now doesn't lead to strange size changes anymore.

- An alternative DVID tile source was added to support its multiscale API.