Toffy

- Updated the `toffy` environment file srivarra ([434](https://github.com/angelolab/toffy/pull/434))
- Ensure safe conversion of panel types during `modify_panel_ranges` alex-l-kong ([443](https://github.com/angelolab/toffy/pull/443))
- Update pulse height generation to be compatible with `mibi-bin-tools==0.2.11` alex-l-kong ([436](https://github.com/angelolab/toffy/pull/436))
- `jupyter_contrib_nbextensions` fix srivarra ([429](https://github.com/angelolab/toffy/pull/429))


📚️ Documentation

- Indicate incompatible normalization notebook with MIBIcontrol 1.8 alex-l-kong ([418](https://github.com/angelolab/toffy/pull/418))
- MIT License srivarra ([448](https://github.com/angelolab/toffy/pull/448))
- Add pre-commit to standardize toffy documentation camisowers ([437](https://github.com/angelolab/toffy/pull/437))
- Cite TSAI in tiling notebooks, deprecating them alex-l-kong ([420](https://github.com/angelolab/toffy/pull/420))

alex-l-kong, camisowers and srivarra

The latest release of `toffy` introduces major updates to the watcher pipeline, removing duplicate data generation, re-extracting incomplete .bin files, and enabling startup without a run folder. It introduces mass proficient image extraction to address storage concerns from large .bin files, round 2 Rosetta for additional denoising, standardized curves and outlier capping for post-Rosetta normalization, and post-reorganization QC metrics to verify across TMAs. Additional features include improved tiled image stitching for TMA + tissue samples, FOV resolution QC checking, and standardized output folders on the D drive for MIBI computers.

🚀 Features

- Check for partially generated fovs camisowers ([416](https://github.com/angelolab/toffy/pull/416))
- Remove hanging `extracted_imgs_dir` reference in Rosetta notebook alex-l-kong ([413](https://github.com/angelolab/toffy/pull/413))
- QC Metrics - Save Longitudinal Transformed Metrics srivarra ([408](https://github.com/angelolab/toffy/pull/408))
- Ensure watcher can wait on run folder to appear alex-l-kong ([401](https://github.com/angelolab/toffy/pull/401))
- QC for TMAs and Control Samples srivarra ([358](https://github.com/angelolab/toffy/pull/358))
- FOV resolution check camisowers ([391](https://github.com/angelolab/toffy/pull/391))
- Adjust rosetta processing paths camisowers ([385](https://github.com/angelolab/toffy/pull/385))
- Tma and tiled run stitching camisowers ([382](https://github.com/angelolab/toffy/pull/382))
- Mass proficient extraction for NAS storage alex-l-kong ([342](https://github.com/angelolab/toffy/pull/342))
- Add Rosetta V2 processing functionality alex-l-kong ([319](https://github.com/angelolab/toffy/pull/319))
- Ensure runs below fovs\_per\_num threshold get skipped during Rosetta t… alex-l-kong ([406](https://github.com/angelolab/toffy/pull/406))
- Skip watcher callbacks for FOVs that already have data generated alex-l-kong ([394](https://github.com/angelolab/toffy/pull/394))
- Ensure incompletely extracted bin files get re-extracted alex-l-kong ([392](https://github.com/angelolab/toffy/pull/392))
- Turn off `combined.csv` file warnings for intermediate callbacks in watcher alex-l-kong ([393](https://github.com/angelolab/toffy/pull/393))
- Turn on y ticks for QC plots alex-l-kong ([386](https://github.com/angelolab/toffy/pull/386))
- Norm workflow alex-l-kong ([361](https://github.com/angelolab/toffy/pull/361))
- Cap normalization at 10X upper bound, and 1X lower bound alex-l-kong ([352](https://github.com/angelolab/toffy/pull/352))
- Add functionality to catch file renaming to watcher alex-l-kong ([359](https://github.com/angelolab/toffy/pull/359))
- Remove warning for .def.hist and .pro.hist files alex-l-kong ([381](https://github.com/angelolab/toffy/pull/381))
- Allow for image specified for flatfield correction to be empty cliu72 ([383](https://github.com/angelolab/toffy/pull/383))

🐛 Bug Fixes

- Fix rescale\_raw\_images error camisowers ([398](https://github.com/angelolab/toffy/pull/398))
- Ensure pulse heights file is named with `_` not `-` alex-l-kong ([412](https://github.com/angelolab/toffy/pull/412))
- Make sure watcher logging actually writes to file on CACs alex-l-kong ([390](https://github.com/angelolab/toffy/pull/390))
- Add a scale factor for `image_stitching.stitch_images` for Photoshop compatibility alex-l-kong ([374](https://github.com/angelolab/toffy/pull/374))

🧰 Maintenance

- Update environment.yml srivarra ([372](https://github.com/angelolab/toffy/pull/372))
alex-l-kong, camisowers, cliu72 and srivarra

The latest release of `toffy` includes some bug fixes to median pulse heights, normalization for all-zero pulse height channels and intermediate callback plotting per FOV, as well as updating multiple tile stitching to be more flexible. In addition a few small bugs were fixed such as missing imports, and updating documentation for installing `pre-commit`. Our GitHub Actions pipeline has also received a light cleanup.

🚀 Features

- GitHub Actions - CI Updates srivarra ([370](https://github.com/angelolab/toffy/pull/370))
- Multiple tile stitching camisowers ([360](https://github.com/angelolab/toffy/pull/360))

🐛 Bug Fixes

- Refresh intermediate callback plotting on each new FOV alex-l-kong ([363](https://github.com/angelolab/toffy/pull/363))
- Fix median pulse height chart drawn by `visualize_mph` alex-l-kong ([356](https://github.com/angelolab/toffy/pull/356))
- Added conda-forge channel to `environment.yml` srivarra ([354](https://github.com/angelolab/toffy/pull/354))
- Ensure normalization doesn't happen for all-zero pulse height channels alex-l-kong ([326](https://github.com/angelolab/toffy/pull/326))
- Update `axes_size` to `axes_font_size` in `3c` notebook alex-l-kong ([343](https://github.com/angelolab/toffy/pull/343))
- Missing import in extraction notebook camisowers ([334](https://github.com/angelolab/toffy/pull/334))

🧰 Maintenance

- Next Release - v0.1.3 srivarra ([371](https://github.com/angelolab/toffy/pull/371))

📚️ Documentation

- Fix brew install command for pre-commit alex-l-kong ([341](https://github.com/angelolab/toffy/pull/341))
- Update README.md ngreenwald ([333](https://github.com/angelolab/toffy/pull/333))
alex-l-kong, camisowers, ngreenwald and srivarra

Adjusts toffy to use the new alpineer repo containing shared utility functions between toffy and ark-analysis. Also adds in a few new features such as tiled image stitching and real time plotting for the watcher notebook. Switches from Travis-CI to GitHub Actions.

🚀 Features

- GitHub Actions srivarra ([328](https://github.com/angelolab/toffy/pull/328))
- Tiled stitching for toffy camisowers ([285](https://github.com/angelolab/toffy/pull/285))
- Flexible Requirements srivarra ([330](https://github.com/angelolab/toffy/pull/330))
- Add ability to override run callbacks to process per FOV alex-l-kong ([313](https://github.com/angelolab/toffy/pull/313))
- Update Rosetta test directory setup alex-l-kong ([300](https://github.com/angelolab/toffy/pull/300))

🐛 Bug Fixes

- Skip empty json files in voltage check camisowers ([290](https://github.com/angelolab/toffy/pull/290))
- Don't display warning if the watcher discovers a temp file alex-l-kong ([329](https://github.com/angelolab/toffy/pull/329))
- Remove MIBItracker tests alex-l-kong ([323](https://github.com/angelolab/toffy/pull/323))
- Make sure Rosetta converts images to `float` prior to running alex-l-kong ([293](https://github.com/angelolab/toffy/pull/293))

🧰 Maintenance

- Conda Environment - scikit-learn \& seaborn srivarra ([297](https://github.com/angelolab/toffy/pull/297))
- Substituted Ark imports with TMI in the Jupyter Notebooks srivarra ([295](https://github.com/angelolab/toffy/pull/295))
- Adding TMI srivarra ([286](https://github.com/angelolab/toffy/pull/286))
- add tmi to environment.yml camisowers ([284](https://github.com/angelolab/toffy/pull/284))
- Image Utils in TMI srivarra ([282](https://github.com/angelolab/toffy/pull/282))
alex-l-kong, camisowers and srivarra

Introduces small bug fixes for loading in panels, extraction crash logic improvements with json files and lossless compression for `tiffs` on by default.

🚀 Features

- Compressed Tiff Files srivarra ([280](https://github.com/angelolab/toffy/pull/280))

🐛 Bug Fixes

- Make sure loading in convert\_panel happens in utf-8 alex-l-kong ([271](https://github.com/angelolab/toffy/pull/271))
- Fix extraction logic for run with a single fov crash camisowers ([269](https://github.com/angelolab/toffy/pull/269))

🧰 Maintenance

- Next Release - v0.1.1 srivarra ([281](https://github.com/angelolab/toffy/pull/281))

📚️ Documentation

- Add link in notebooks for the ark tiled Image Stitching script camisowers ([278](https://github.com/angelolab/toffy/pull/278))
- Toffy Setup Video Fix srivarra ([267](https://github.com/angelolab/toffy/pull/267))
alex-l-kong, camisowers and srivarra

First release of the beta version of **toffy**. This includes new MIBI image generation and processing tools, organized into a multistep pipeline for users to follow. Also contains adjustments made in preparation for and following the [MIBI Workshop](https://www.angelolab.com/mibi-workshop).


🚀 Features

- Notebook 1 normalization checks camisowers ([250](https://github.com/angelolab/toffy/pull/250))
- Add \_pulse\_height.csv file generation to watcher camisowers ([262](https://github.com/angelolab/toffy/pull/262))
- Custom error for Rosetta file copying camisowers ([258](https://github.com/angelolab/toffy/pull/258))
- Json file encoding camisowers ([243](https://github.com/angelolab/toffy/pull/243))
- Stitch images function without run file camisowers ([237](https://github.com/angelolab/toffy/pull/237))
- Toffy notebook fixes camisowers ([230](https://github.com/angelolab/toffy/pull/230))
- Create release-drafter.yml srivarra ([223](https://github.com/angelolab/toffy/pull/223))
- Rosetta stitching fix camisowers ([227](https://github.com/angelolab/toffy/pull/227))
- Add interactive tiled region overlay alex-l-kong ([86](https://github.com/angelolab/toffy/pull/86))
- Add temporary `scripts` directory to track notebook updates alex-l-kong ([220](https://github.com/angelolab/toffy/pull/220))
- Rosetta notebook testing updates camisowers ([212](https://github.com/angelolab/toffy/pull/212))
- Optimize Watcher Tests ackagel ([219](https://github.com/angelolab/toffy/pull/219))
- Remove `moly_path` requirement for `remap_to_reorder_fovs` alex-l-kong ([215](https://github.com/angelolab/toffy/pull/215))
- Update run metrics saving behavior camisowers ([213](https://github.com/angelolab/toffy/pull/213))
- Add checks for out-of-bounds coordinates during tiling process alex-l-kong ([209](https://github.com/angelolab/toffy/pull/209))
- Panel conversion implementation camisowers ([202](https://github.com/angelolab/toffy/pull/202))
- Added `.DS_Store` in the `.gitignore` srivarra ([206](https://github.com/angelolab/toffy/pull/206))
- qc metrics violin plots on normal tissues mheskett ([153](https://github.com/angelolab/toffy/pull/153))
- Ensures co-registration path is absolute alex-l-kong ([196](https://github.com/angelolab/toffy/pull/196))
- Image stitching in watcher camisowers ([152](https://github.com/angelolab/toffy/pull/152))
- Adjust min TMA size to 2x2 alex-l-kong ([194](https://github.com/angelolab/toffy/pull/194))
- Simplify QC metric visualization procedure alex-l-kong ([174](https://github.com/angelolab/toffy/pull/174))
- Output float32 images for normalization notebook camisowers ([155](https://github.com/angelolab/toffy/pull/155))
- run fovs in numerical order camisowers ([154](https://github.com/angelolab/toffy/pull/154))
- Check voltage camisowers ([148](https://github.com/angelolab/toffy/pull/148))
- Split run file camisowers ([145](https://github.com/angelolab/toffy/pull/145))
- Update QC notebook with real data camisowers ([147](https://github.com/angelolab/toffy/pull/147))
- Extract bin notebook camisowers ([130](https://github.com/angelolab/toffy/pull/130))
- Mph plots camisowers ([101](https://github.com/angelolab/toffy/pull/101))
- Mph histograms camisowers ([126](https://github.com/angelolab/toffy/pull/126))
- Implement json helper funcs mheskett ([142](https://github.com/angelolab/toffy/pull/142))
- Json helper functions mheskett ([138](https://github.com/angelolab/toffy/pull/138))
- Small tweaks ngreenwald ([137](https://github.com/angelolab/toffy/pull/137))
- Format data for analysis ngreenwald ([131](https://github.com/angelolab/toffy/pull/131))
- added moly\_path option srivarra ([114](https://github.com/angelolab/toffy/pull/114))
- Mph norm workflow ngreenwald ([125](https://github.com/angelolab/toffy/pull/125))
- find moly FOVs based on JSON file ngreenwald ([124](https://github.com/angelolab/toffy/pull/124))
- Rosetta simplification ngreenwald ([107](https://github.com/angelolab/toffy/pull/107))
- implements watcher chaining architecture ackagel ([99](https://github.com/angelolab/toffy/pull/99))
- Rosetta modifications ngreenwald ([92](https://github.com/angelolab/toffy/pull/92))
- Rename warning camisowers ([103](https://github.com/angelolab/toffy/pull/103))
- Update repo setup for watcher ngreenwald ([93](https://github.com/angelolab/toffy/pull/93))
- Rename fov folders camisowers ([85](https://github.com/angelolab/toffy/pull/85))
- Streak Detection srivarra ([71](https://github.com/angelolab/toffy/pull/71))
- adds fov\_watcher template ackagel ([53](https://github.com/angelolab/toffy/pull/53))
- Don't check contrast for skimage save function. srivarra ([58](https://github.com/angelolab/toffy/pull/58))
- Add ROI name prefix to tiled region FOV names alex-l-kong ([66](https://github.com/angelolab/toffy/pull/66))
- Rosetta workflow ngreenwald ([43](https://github.com/angelolab/toffy/pull/43))
- Write co-registration settings to JSON param file alex-l-kong ([60](https://github.com/angelolab/toffy/pull/60))
- Add watcher compatible QC notebook alex-l-kong ([50](https://github.com/angelolab/toffy/pull/50))
- Add notebook to change co-registration parameters in settings.py alex-l-kong ([45](https://github.com/angelolab/toffy/pull/45))
- Add tiled regions notebook to toffy alex-l-kong ([41](https://github.com/angelolab/toffy/pull/41))
- Normalize images by MPH ngreenwald ([35](https://github.com/angelolab/toffy/pull/35))
- Change x to columns and y to rows in TMA script alex-l-kong ([39](https://github.com/angelolab/toffy/pull/39))
- Add TMA FOV validation functions for user sanity checking alex-l-kong ([30](https://github.com/angelolab/toffy/pull/30))
- Add TMA tiling utils into creed alex-l-kong ([13](https://github.com/angelolab/toffy/pull/13))
- Support non-rectangular TMAs ngreenwald ([19](https://github.com/angelolab/toffy/pull/19))
- Automatic checking of hashes for transferred files ngreenwald ([21](https://github.com/angelolab/toffy/pull/21))
- Add automatic detector sweep ID ngreenwald ([17](https://github.com/angelolab/toffy/pull/17))
- Add Rosetta matrix functionality ngreenwald ([27](https://github.com/angelolab/toffy/pull/27))
- Add coveralls support alex-l-kong ([14](https://github.com/angelolab/toffy/pull/14))
- Add Travis and pytest support alex-l-kong ([15](https://github.com/angelolab/toffy/pull/15))
- Add notebooks ackagel ([3](https://github.com/angelolab/toffy/pull/3))

🐛 Bug Fixes

- get\_max\_img\_size works with subdir camisowers ([264](https://github.com/angelolab/toffy/pull/264))
- Rosetta mass 39 naming conflict camisowers ([252](https://github.com/angelolab/toffy/pull/252))
- Panel conversion column typo camisowers ([247](https://github.com/angelolab/toffy/pull/247))
- Ignore stitched\_images folder in rosetta processing camisowers ([239](https://github.com/angelolab/toffy/pull/239))
- Ensure path to `rosetta_matrices` is added in the setup notebook alex-l-kong ([236](https://github.com/angelolab/toffy/pull/236))
- Catches tempdir cleanup error camisowers ([225](https://github.com/angelolab/toffy/pull/225))
- Make watcher skip moly points ackagel ([117](https://github.com/angelolab/toffy/pull/117))
- Remove endline in column names from correctly formatted panel camisowers ([211](https://github.com/angelolab/toffy/pull/211))
- Fix pulse\_height\_dir path in normalize notebook alex-l-kong ([210](https://github.com/angelolab/toffy/pull/210))
- Watcher hidden files camisowers ([193](https://github.com/angelolab/toffy/pull/193))
- Added the location of the Optical Image File in the Notebook srivarra ([182](https://github.com/angelolab/toffy/pull/182))
- Empty json check camisowers ([187](https://github.com/angelolab/toffy/pull/187))
- Make sure `mapping_path` load call gets assigned to `mapping` prior to relabeling TMA FOV names alex-l-kong ([186](https://github.com/angelolab/toffy/pull/186))
- Adjusted Streak Detection mask generation parameters srivarra ([119](https://github.com/angelolab/toffy/pull/119))
- Fix import error camisowers ([96](https://github.com/angelolab/toffy/pull/96))
- Add scheme for renaming TMA manual FOV duplicates alex-l-kong ([84](https://github.com/angelolab/toffy/pull/84))
- Ensure JSON data is read in with encoding UTF-8 alex-l-kong ([36](https://github.com/angelolab/toffy/pull/36))
- Change notebook import from creed to toffy alex-l-kong ([33](https://github.com/angelolab/toffy/pull/33))
- Update repo for toffy ngreenwald ([32](https://github.com/angelolab/toffy/pull/32))

🧰 Maintenance

- Adjusted requirements to work with ark-analysis on Python 3.7 srivarra ([132](https://github.com/angelolab/toffy/pull/132))
- Pin ark requirement ngreenwald ([121](https://github.com/angelolab/toffy/pull/121))
- Updating the License srivarra ([47](https://github.com/angelolab/toffy/pull/47))

📚️ Documentation

- Add workshop links to readme camisowers ([260](https://github.com/angelolab/toffy/pull/260))
- Update flowchart in readme camisowers ([248](https://github.com/angelolab/toffy/pull/248))
- Update README.md with link to discussions ngreenwald ([229](https://github.com/angelolab/toffy/pull/229))
- Reorg notebook cleaning camisowers ([222](https://github.com/angelolab/toffy/pull/222))
- Add documentation for TMA notebook alex-l-kong ([207](https://github.com/angelolab/toffy/pull/207))
- Update readme with directory structure camisowers ([190](https://github.com/angelolab/toffy/pull/190))
- Added Step 3e to the pipeline flowchart srivarra ([204](https://github.com/angelolab/toffy/pull/204))
- Notebook 4a - Documentation Improvements srivarra ([177](https://github.com/angelolab/toffy/pull/177))
- Toffy Pipeline Flowchart srivarra ([200](https://github.com/angelolab/toffy/pull/200))
- Added the location of the Optical Image File in the Notebook srivarra ([182](https://github.com/angelolab/toffy/pull/182))
- Notebook 1 - Documentation Improvements srivarra ([175](https://github.com/angelolab/toffy/pull/175))
- 3b and 3d documentation updates camisowers ([176](https://github.com/angelolab/toffy/pull/176))
- Add MPH description ngreenwald ([191](https://github.com/angelolab/toffy/pull/191))
- Add updated documentation for QC notebook alex-l-kong ([171](https://github.com/angelolab/toffy/pull/171))
- Add brief explanation for why the normalize notebook is significant alex-l-kong ([173](https://github.com/angelolab/toffy/pull/173))
- Added Git Installation Instructions for CAC srivarra ([178](https://github.com/angelolab/toffy/pull/178))
- Detailed panel description ngreenwald ([136](https://github.com/angelolab/toffy/pull/136))
- Added blurb about Python Version srivarra ([118](https://github.com/angelolab/toffy/pull/118))
- toffy documentation standardization ngreenwald ([77](https://github.com/angelolab/toffy/pull/77))
- Install instructions camisowers ([67](https://github.com/angelolab/toffy/pull/67))
- Toffy rename ngreenwald ([26](https://github.com/angelolab/toffy/pull/26))
ackagel, alex-l-kong, camisowers, mheskett, ngreenwald and srivarra