_Multimodal inputs:_
- Multiple and linear variable inputs can be used to train models in addition to images, using the input_header parameter. Multiple inputs are merged as a concatenated vector with post-convolutional layers prior to hidden dense layers
- Models can also be trained without images with the hyperparameter drop_images
_CPH models:_
- Added negative_log_likelihood loss for time to event models. The outcome must be set to the time to event. The presence or absence of event numerical variable should be included as the first element in the input_header. This replicates DeepSurv i.e. neural network log loss cox proportional hazards (CPH) model. Also included custom metric c_index for tracking training and for evaluation of CPH models.
- Included permutation feature importance as an option for evaluate. When set to true, will run permutation feature importance to define the relative benefit of histology and each input in input_header.
_CLAM support:_
- Added CLAM training via new pipeline function SFP.train_clam().
- Added ability to export whole-slide, tile-level feature activations to torch format (for use with CLAM): AV.export_to_torch()
- Gave ActivationsVisualizer a min_tiles_per_slide (necessary for CLAM)
- Preliminary support for CLAM evaluation (SFP.evaluate_clam())
_QOL improvements / new features:_
- sf.SlideflowProject -> sf.Project
- Cleaned up method of instantiating a project; interactive project creation can be done by calling sf.Project.from_prompt(), but regular initialization now acts as expected without prompting for user input
- Moved several project creation functions out of the Project class and into project_utils
- Multiple categorical outcomes can now be used as model output by supplying a list of headers to outcome_label_headers() in SFP.train() or evaluate()
- Removed multi_outcome argument from SFP.train()
- Added slide PIL image error handling (with corrupted SVS files)
- Models no longer stored in .h5 format; saved using TF SavedModel format
- update.py now includes ability to update old models from .h5 to newer format
- Default image storage format for tfrecords is now lossless PNG
- Can now specify PNG or JPG tile extraction with img_format argument
- Unified & automatic tfrecord parsing, auto-detects internal tfrecord format for improved compatibility
- Training augmentation now includes random compression with JPG encoding at random quality levels
- Models now copy hyperparameters.json & slide_manifest.log into the model folder
- Adds roi_method argument to generate_heatmaps()
- added support for manual k-folds with k-fold-manual strategy and k_fold_header
- Can randomize origin during tile extraction with randomize_origin argument
- Standardized color output with heatmaps to display results 0 -> 1
- Added predict() functionality to SlideReader, which generates a slide-level grid of predictions / activations from a model
- Added SFP.predict_wsi() which generates whole-slide grid predictions from a model
- New argument for SFP.train(): skip_metrics. If True, will skip the validation metrics generation during training.
- For generate_activations, can now specify include_logits = True|False.
- Can now manually specify which predictions to save during evaluation - 'tile', 'patient', 'slide', or True for all
- relative paths now used by default in projects; specified with $ROOT/ prefix in settings.json
- Updated test suite & added new predict_wsi test
- TFRecords will now store tile location. Some pipeline functions (e.g. CLAM attention heatmaps) will require new TFRecords in this new format
- Added learning_rate_decay and dropout hyperparameters
- Streamlined ModelActivationsInterface; activation layers can now be manually specified and legacy compatibility is maintained
- Added support for NOT passing a slide manifest to SlideflowModel
- Commented-out code for preliminary RNA-seq support (as outcome)
- Adds sf.experimental for features not (yet) fully fleshed-out
- Removed use_fp16 references and replaced with project-wide mixed_precision setting
- Documentation updates
- Bug fix in ModelActivationsInterface with legacy models
- Removed GPU pools & autoselection
- Removed multithreading for predict_wsi()
- Replaces internal logging functions with official python logging module
- Logging improvements -> moved many messages to DEBUG logging level, which prompt with dim text
- Overhaul of logging, progress bars, and user experience
- New sf.io.reader submodule includes tensorflow-agnostic tfrecord reader, using dareblopy backend, for pytorch compatibility
- New optional tfrecord sharding
- SlideReader, when manually called, no longer automatically skips slides without ROI
- Heatmap now reads tile_px/tile_um from model, does not require explicitly setting
_Performance improvements:_
- 100x+ improvement in evaluation and validation speed
- 10-fold improvement in tile extraction speed with multiprocessing extraction
- Multi-GPU training can be enabled by passing multi_gpu=True to SFP.train() (SFP must be configured to have multiple GPUs visible via CUDA)
- Validation histogram generation & CSV predictions disabled during training
- Heatmaps can auto-buffer slides for faster generation
- Improved heatmap thumbnail resolution & speed of generation
_Breaking changes:_
- Annotation 'outcomes' are now referred to as annotation 'labels', since slide level features may be used as either input or outcomes
- All instances of "outcome_header" arguments replaced with "outcome_label_headers"
- Renamed AV.export_activations_to_csv() -> export_to_csv()
- Renamed SFP.export_activations_analytics() -> generate_activations()
- sf.SlideflowProject -> sf.Project
- Pipeline functions no long accept specific validation variables as arguments. To create and use a different validation plan than default, use sf.project.get_validation_settings() and pass the returned namespace to SFP.train() via `validation_settings`
- Removed validation settings from project settings
- Removed "use_fp16" from project-wide settings, replaced with "mixed_precision"
_General code cleanup:_
- Removed unused dependencies
- Removed old commented-out or redundant sections
- Re-ordered SFP functions alphabetically
- Improved indentations, arguments formatting
- Added error logging instead of sys.exit
- Moved SlideflowProject -> sf.project
- Moved multiprocessing SFP utilities into sf.project_utils
- Optimized import speed; tensorflow only imported when needed
- Reworked statistics module to reduce redundant code
- get_labels_from_annotations() now accepts a 'key' variable to customize the returned dictionary
- get_training_and_validation_tfrecords() now requires 'outcome_key' variable to specify the dictionary key that includes the outcome variables, provided in slide_labels_dict
_Bug fixes:_
- Bug fix in heatmaps (tiles were being shuffled prior to display)
- Bug fix for SFP.extract_tiles_from_tfrecords()
- fixed Dataset() global variables bug
- fixed bug where get_manifest() from dataset was not using filters
- fixed bug with PDF image error handling
- Bug fix with large datasets in Dataset.get_tfrecords()
- Bug fix with realtime normalization
- Fixed bug where evaluate() was not loading already-assigned int labels for multi-categorical outcomes.
- Bug fix where patient-level predictions were overwriting slide-level predictions
- Bug fix with permutation_feature_importance
- Fixes in interleave_tfrecords
- Progress bar bug fixes
- NASNetLarge bug fix
- Removes 'auto' num_thread option for Heatmap, which was unstable
- Mosaic logging fix
- Activations fix for handling empty tfrecords
- Other various fixes
New augmentation function - JPEG and/or flipping or rotation
- Augmentations can be specified by passing string to augment argument
- If 'j' is in string, applies random JPEG compression augmentation
- If 'x' is in string, applies random horizontal flip
- 'y' applies random vertical flip
- 'r' applies random rotation
- True (or 'xyrj') applies all augmentations
_New dependencies:_
- torch
- torchvision
- lifelines