Python-catmaid

Contributors: Albert Cardona, Tom Kazimiers, Andrew Champion


Notes

- A virtualenv update required!

- The 'classic' node provider has been removed, use 'postgis3d' or 'postgis2d'
instead.

- Performance of the default node query strategy (NODE_PROVIDER = 'postgis3d')
improves now when connection pooling is used and PREPARED_STATEMENTS = True.

- Both 'postgis2d' and 'postgis3d' node providers support now prepared
statements for connector queries.

- PyPy is now supported and can be used to improve performance of back-end heavy
endpoints. Most functionality is available, except for the following: Ontology
clustering, Cropping, Synapse clustering, HDF 5 tiles and User analytics. To
use PyPy, a new virtualenv using the PyPy executable has to be created.

- There is now a catmaid_prewarm_db management command, which can be used to
populate OS and database buffers with heavily used tables.


Features and enhancements

Graph widget:

- The "Selection" tab now has two new buttons that provide the ability to invert
the selection, and also to select graph nodes based on matching a text string
or by a regular expression (when the text starts with a '/').

3D Viewer:

- With the help of the new "Update active" checkbox in the "View" tab it is now
possible to automatically update the active skeleton if it changes.

- It is now possible to hide edges that represent a link between a node and a
connector. This is also respected with connector restrictions enabled and can
be set with new "Show connector links" checkbox in the "View settings" tab.

Selection table:

- To prevent accidental sorting the new "Lock order" checkbox can be enabled.
Clicks on the table headers will have no effect if the order is locked.

- Two new action icons are added on the right of each row: move a skeleton up or
down in the table.

Annotation graph:

- A new 'Annotation Graph' widget provides a graph like visualization of
CATMAID's annotation space, which is mainly useful to visualize annotation
hierarchies. Meta-annotation are linked to annotations through edges. This
widget can be opened with Ctrl+Space together with the 'annotation-graph'
keyword.

- Optionally, the minimum and maximum number of linked annotations in a single
hierarchy can be configured.

- Selecting an annotation will query all (sub-)annotated skeletons, which can be
used as through a regular skeleton source selection in other widgets.

Miscellaneous:

- Tracing tool: the "Show labels" setting is now remembered across sessions.

- Right clicking on a stack viewer won't bring up the browser context menu
anymore. Instead, the right mouse button can now be used for panning, too.

- The merge dialog has now a "Swap" button in the lower left corner. It can be
used to swap winning and losing skeleton in a merge.

- Add a 'select' tracing modes to the tracing tool: Clicking an active skeleton or
synapse tracing mode button (first two buttons in toolbar) will switch to the
select mode where clicking doesn't create new nodes.

- The layer settings of image data layers allow now to add custom mirrors,
which is useful to e.g. make a local copy of the image data availabl in
CATMAID. This can reduce latency due to remote image loading considerably. See
the user FAQ in the documentation for more details.

- Annotation pages in the Neuron Navigator now also list meta annotations and
provide a way to de-annotate them.


Bug fixes

- Fix occasional tracing overlay resizing problem where a portion of the screen
would not show tracing data, but also won't update.

- 3D viewer: broken sections can now be displayed again.

- 3D viewer: the Z plane with stack images will now render correctly for the
default maximum zoom level.

Contributors: Chris Barnes, Albert Cardona, Andrew Champion, Tom Kazimiers

Notes

- A new NODE_PROVIDER option can be added to settings.py, which can be used to
configure the way node queries are executed. Options are 'classic',
'postgis3d' and 'postgis2d'. Depending on your environment, 'postgis2d' might
be fastest, but 'postgis3d' is the current default (i.e. what has been used so
far).

- The new PREPARED_STATEMENTS option can be added to settings.py and set to
True to improve node query performance when connection pooling is used.
Without connection pooling, this setting hurts performance, which is why it is
set to False by default.

- Stack mirrors (see below) are not displayed by default in CATMAID's standard
data views. To show them, '"show_mirrors": true' (including double quotes) has
to be added to the respective data view configuration in CATMAID's admin
interface.

Features and enhancements

Stack mirrors:

- To reduce loading time of image data from different places around the world,
it is common to copy the image data set and make it available from a server
closer to its users. These "stack mirrors" are now handled differently by
CATMAID. Until now it was common to select the closest mirror manually by
clicking on an image on CATMAID's project list or by clicking on the
respective link. To select a particular "stack mirror" one now only links can
be used, individual images are not shown anymore. If no images were used
before and only links were shown on CATMAID's home page, only small visual
change was mode: an extra stack link on top of all mirrors is added. As an
alternative to selecting individual mirrors, both, images and stack links can
now be used to open a stack as well and CATMAID selects the fastest mirror
automatically.

- When using links to a particular view in CATMAID ("URL to this view"), CATMAID
will automatically select the fastest available stack mirror.

- Once a stack is displayed the stack mirror in use can be changed: After
clicking on the white-on-blue square in the lower left corner of the image
data display, the image display settings will show up. In there the active
mirror can be selected from a drop-down list.

Treenode Viewer:

- A new tool similar to the connector viewer, which allows users to
quickly view the nodes in a treenode table.

- The viewer is opened from the treenode table.

- Nodes are shown only if they are filtered in the table, and appear in
the order that they appear in the table.

- When the sorting and filtering in the table is changed, users can
refresh the viewer from the table to reflect these changes.


Connector Viewer enhancements:

- Connectors can now be constrained by their pre-synaptic and post-synaptic
skeletons.

- If either is unpopulated, it is unconstrained on that side (i.e. all
outgoing connectors can be found by clearing the 'Post- skeletons' set).

- Sorting algorithms based on treenodes associated with the connector
(e.g. depth) can be applied either to presynaptic or postsynaptic
skeletons.

- The 'Reverse' button switches the contents of the two skeleton sets.

- The 'Sync' checkbox sets the pre- and post- synaptic sets to have the
same contents (the union of their contents when the box is checked)
and keeps them that way: this is useful for inspecting all connectors
acting within a set of skeletons, such as reciprocal connections.

- For undirected connectors (gap junctions, abutting etc.), you can use
either the 'pre' or the 'post' set. If both sets are populated, only
connectors touching a pre- skeleton and a post- skeleton (with 2
distinct edges) will be shown.

- When using the connector viewer as a skeleton source, the output is the
union of the two skeleton sets.

- The controls have been separated out into tabs.

- N.B.: Because the Connector Viewer is now populated in a different way
to the Connector Table, you can now only open a Table from a Viewer if
one of the skeleton sets is empty, and a connector type other than 'All'
is selected.


Miscellaneous:

- Bookmarks are now persistent for each project.

- The Graph Widget now offers two options when exporting SVGs: regular ones and
Adobe Illustrator compatible ones (because Adobe isn't SVG standard conform).
An option dialog is shown after clicking "Export SVG".

- From the Graph Widget exported SVG files preserve now the view of the widget
(zoom and pan).

- The tag table can now be constrained by a set of skeletons.

- The tag table now gives the user information on how many skeletons are
being used as constraints, and how many tags/skeletons/nodes are
selected.

- When the tag table is refreshed, the filters and the last sort are
persisted.

- The treenode table can now be filtered by node confidence, creator and
reviewer.

- When creating new volumes with the help of the Volume Manager, the color and
opacity of the 3D preview can now be adjusted.


Bug fixes

- 3D Viewer: custom tag highlighting now also works for tags with upper case
letters.

- Layer settings: checkboxes now have correct default value.

- Tag table: any tags with identical names are now treated as identical.

- Tag table: fixed off-by-one error in node count.

Contributors: Chris Barnes, Andrew Champion, Tom Kazimiers

Notes:

- The h5py dependency has been removed. If tile source tile 3 should be used in
a new instance, the h5py library has to be installed manually.


Features and enhancements

Connector Viewer:

- A new tool used to view the connectors associated with a skeleton set
of skeletons.

- Shows a grid of mini stack viewers with their own tracing overlays,
focused on connectors associated with the skeleton by a user-selected
relation (i.e. outgoing, incoming, gap junction or other).

- Connectors can be sorted by their absolute or proportional depth on
their respective skeleton trees, by the connector ID or by the
skeleton name.

- Mini stack viewers can inherit settings from a user-defined main stack
viewer; users can focus the main stack viewer on any connector by
clicking on its ID in the mini stack viewer title bar.

- Users can open a connector table from a connector viewer and vice
versa.

- Accessible with Ctrl+Space 'connector-viewer'.


Connectivity table:

- Original colors of skeletons added to a Connectivity Table can now optionally
be used when using its skeletons in other widgets (new checkbox: "Original
color").

- The colors used in the Connectivity Graph Plot are now the ones assigned to the
input skeletons, i.e. the ones selected in the Connectivity Widget. These in
turn are either its input skeleton color or the default skeleton color.

- Manual re-ordering of partner count columns has been removed. Neurons can now
be re re-ordered through the list at the top with the help of two icons in
each row (up and down).

- All partner tables are now paginated with a default page size of 50 entries.
The page size can be adjusted for all tables at the same time with a drop-down
menu at the top. "Select all" check-boxes select all entries across all pages.


Graph widget:

- Active Graph Widget windows will now zoom in smaller steps if the Shift key
is pressed.

- Graphs saved to a JSON file now keep visibility information. Loading them
will load all nodes hidden that have been hidden when saving the file
initially..


3D Viewer

- Meshes are replaced with volumes and the "Show meshes" checkbox has been
removed. Existing meshes have been transformed into volumes.

- The new coloring mode "Last Reviewer" will color skeleton nodes by the user
color of the user that reviewed them last. The "User colormap" button can be
used to show the mapping from color to user name.


Miscellaneous:

- The Selection Table can now be opened through Ctrl+Space by using the handle
'selection-table' instead of 'neuron-staging-area'.

- Project and stack menu entries are now sorted by default, which makes it easier
to deal with many projects and stacks.

- Node bookmarks can now be removed through an X icon in their respective row in
the bookmark dialog.

- Skeleton Analytics in the Review Widget will now identify nodes in broken
sections of all stacks linked to the current project.

- The front page project list can now be loaded as a widget using Ctrl-Space
with 'project-list'.

- Synaptic fractions: colors of groups and partners can now be changed through
legend of graph.

- The tag table now has a refresh button to manually update the cache and
redraw the table.

- The tag table can now open a treenode table pre-filtered for nodes with
the selected labels.


Bug fixes

- The size of the embedded 3D viewer in the split/merge dialog is now adjusted
dynamically based on the available space. This prevents some problems with
lower display resolutions.

- 3D viewer: loading of box volumes workes again.

- The statistics widget back-end will now correctly use up-to-date time-zone
information for the query time range. This fixes some time zone conversion
corner cases.

- Graph widget: SVG export works again. The exported SVG now groups labels with
node/edge. Now also single node exports are supported.

- If Ctrl-Z was pressed and released very quickly, two undo steps were taken.
Now always only one undo step will be done per single Ctrl-Z click if not hold
down.

- 3D viewer: partner node spheres are now also shown for restricted connectors.

- Multiple stack viewers no longer use the same ID to make requests to
the database.

- The connector table can now be used as a skeleton source, where
previously an error would be raised.

- The tag table now collapses any tags with identical names, but as a
consequence does not show the tag ID (as there may be multiple IDs)

Contributors: Chris Barnes, Andrew Champion, Tom Kazimiers

Notes

- Virtual env update required

- Postgres 9.6 is now supported.

Features and enhancements

3D Viewer:

- Shift+Click can now be used to jump to arbitrary locations on rendered
skeletons. Objects already clickable in the past (like tags or the soma
spheres) are still given precedence.


Tag Table:

- New widget allowing users to get information about node label usage in
the project, and select skeletons based on labels which their nodes
have.

- Accessible with Ctrl+Space by the name 'tag-table' or through the 'T' widget
button in the Tracing Tool.


Measurement Table:

- The neuron name is now quoted in the CSV file export. This makes it more
robust when commas are used in the name representation (e.g. if annotations).

- XLSX spreadsheet export is now supported through an "Export XLSX" button.


Importer:

- Add support for importing ontology and classification information along with
projects, stacks and stack groups in project file based import ("Image data
importer" in admin view).

- Projects to be imported can be split across multiple documents, if local or
remote file import is used. This helps organizing image collections manually.


Tracing tool:

- Holding P will show the active skeleton in all open 3D viewers.

- The semicolon (";") key no longer switches to skeleton tracing mode, because
it is active by default.

- Bookmarks can be set by pressing semicolon (";") and then a marker key, which
will mark the active node (or location if no node is active) with that key.
Pressing backtick ("`") followed by the marker key will return to the marked
node or location. Pressing single quote ("'") followed by the marker key will
instead activate the marked skeleton and move to the closest skeleton node.

- Shift + H will go to the last node you edited in any skeleton.


Tracing layer:

- Connectors connected to the active skeleton are now colored with a distinct,
more yellow hue of orange, even if the connection is not in the visible
section.


Miscellaneous:

- The Graph Widget can handle larger graphs much better now. Also its mouse
action have slightly changed: moving while having the left mouse button down
will pan the view, pressing additionally the Ctrl key enters rectangular
selection mode.


Deprecations and Removals

- Special behavior of tile source type 2 stacks to interoperate with a Volumina
tile server prototype has been removed.


Bug fixes

- Review widget: auto centering works again

- Review widget: location lookup of warnings now works again in all cases.

- Skeleton source subscriptions: fix accidental sharing of skeleton sources.
This caused e.g. losing the connection between a Selection Table and a 3D
viewer as soon as another 3D viewer was opened.

- The Classification Editor of CATMAID's Ontology Tool can be started again.

- Using the closing bracket ("]") to walk a neuron does not stop working
anymore after the end of a branch is reached.

- 3D viewer: finding the date of the most recent change among all skeletons
considered in a history animation now works as expected. The end point of the
animation is now calculated correctly and all changes can be seen.

- The Classification Clustering Widget can be opened again.

Contributors: Gregory Jefferis, Tom Kazimiers

Notes

- Virtualenv update required

Features and enhancements

Tracing layer:

- Alt+Click so far created gap junction connectors. This behavior is replaced
with opening a small context menu that provides access to all currently
available connector types (abutting, gap junction, presynaptic, postsynaptic).
Optionally, through the Settings Widget, remembering the last connector type
created can be enabled. If this is the case, regular Shift+Click behavior is
to create the last connector type, as opposed to pre/post-synaptic connectors.


3D viewer:

- History replay now has a reset button to load the most recent version of each
skeleton. The new pause button will stop the animation at the current time and
pressing start will resume playback. Additionally, a slider control now allows
to move to arbitrary points in time of the history animations. Touching the
slider will automatically pause the replay.


Neuron dendrogram:

- Controls are now organized in tabs and a new setting was added to change the
line width.


Miscellaneous:

- If users have sufficient permissions, they can click on individual names in
the Project Statistics Widget to show a User Analytics window for the
respective user and currently selected time frame.

- Pressing Ctrl+Space will now open a dialog which allows opening a widget based
on its short name (e.g. neuron-dendrogram) or parts of it.

- A small system check widget has been added, mainly useful for development. It
currently only supports FPS measurement and can be opened by pressing F6 and
entering "system-check".


Bug fixes

Contributors: Andrew Champion, Tom Kazimiers


Features and enhancements

3D viewer:

- Picking now respects the Z plane. if displayed. Shift+click any location on
it and the stack viewer position is changed accordingly. This works just like
picking nodes, except that the active node doesn't change.

- The original camera location is now restored by default after exporting an
animation.

- A new tab called 'History' gives access to tools for replaying the
reconstruction of the currently loaded skeletons. The time range as well as
the time advance per frame can be adjusted. By default, the time range covered
starts ith the first change and ends with the last one (when no custom date is
entered). Optionally, empty bouts where no changes happend will be skipped if
a length (in minutes) is specified. If not disabled through the 'Include
merges' checkbox, arbors that only where merged in at one point will also be
displayed before such a merge. Tags on individual nodes are currently hidden
during animation.


Review widget:

- Skeleton Analytics, which was a separate widget before, is now available
through a new tab in the Review Widget. It still lists the same problems for
selected neurons as before (i.e. missing end tags).


Connector table (available through Neuron Navigator):

- Support for abutting connectors and gap junctions has been added.

- The displayed table can now be exported as CSV file.

- The listed information for connector relations with only one partner is now
consistent with multi-relation cases: the linked node ID, node location and
skeleton ID are shown instead of the connector location along an empty node
and skeleton fields.

- The widget can now pull skeletons to list connectors for from other skeleton
sources. It acts itself as a skeleton source as well and provides all result
skeleton IDs.


Importer:

- The project and stack importer now supports the customization of when projects
are considered known as well as what to do with known projects: ignore, merge
or replace.

- If remote hosts are used as a project or stack import source, HTTP
Authentication can now be used to get access to a server.


Miscellaneous:

- The tracing tool has a new icon.

- If users get a permission denied error (e.g. due to being logged out or just
having not enough permissions), a login dialog is now shown. It allows users
to re-login or change the user entirely. The action causing the permission
error, is *not* repeated automatically.

- Using brackets to navigate along a skeleton now also works if the
reconstruction data is hidden (e.g. when the space bar is held down).


Bug fixes

- The statistics widget now properly respects time zones when grouping by day.

- Partner neurons listed in the Connectivity Widget can now be filtered again
with regular expressions (when the filter pattern starts with "/").

- Loading stack groups through URLs now correctly respects the specified location.

- H works correctly when a virtual node is active.

- Pressing E in the Review widget now works again when a segment was completely
reviewed.

- The group membership permission tool in CATMAID's admin area works with groups
again.