Jwql

==================

Web Application
~~~~~~~~~~~~~~~
- Hover over proposal thumbnails by shanosborne in https://github.com/spacetelescope/jwql/pull/927
- Not found images replaced with default image by BradleySappington in https://github.com/spacetelescope/jwql/pull/949
- Remove second set of colorbar tick labels from preview images by bhilbert4 in https://github.com/spacetelescope/jwql/pull/963
- Split program page by obsnum by bhilbert4 in https://github.com/spacetelescope/jwql/pull/948
- Interactive preview images by BradleySappington in https://github.com/spacetelescope/jwql/pull/965
- Alert user to non-existant proposal number by bhilbert4 in https://github.com/spacetelescope/jwql/pull/994
- Fix bad thumbnails by bhilbert4 in https://github.com/spacetelescope/jwql/pull/997
- Explore by group type by BradleySappington in https://github.com/spacetelescope/jwql/pull/1000
- Default scaling params for preview images where all pix are NaN by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1006
- Explore int grp difference by BradleySappington in https://github.com/spacetelescope/jwql/pull/1020
- Add inst select home search by mengesser in https://github.com/spacetelescope/jwql/pull/1010
- Allow proposal thumbnails to come from dark thumbnails by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1001
- Adding Next and Previous Buttons by mfixstsci in https://github.com/spacetelescope/jwql/pull/978
- Build archive page from database by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1030
- Select anomaly name formatting by rcooper295 in https://github.com/spacetelescope/jwql/pull/1053
- Implement "Viewed" Button and Filtering by BradleySappington in https://github.com/spacetelescope/jwql/pull/1049
- fixes for no data found on server by penaguerrero in https://github.com/spacetelescope/jwql/pull/1042
- Filter archive page by BradleySappington in https://github.com/spacetelescope/jwql/pull/1065
- Exposure page: Default to show rate img, keep order of radio buttons constant by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1111
- Filter type by BradleySappington in https://github.com/spacetelescope/jwql/pull/1092
- Sort obs date by BradleySappington in https://github.com/spacetelescope/jwql/pull/1113
- added scattered light, light saber anomalies for niriss by rcooper295 in https://github.com/spacetelescope/jwql/pull/1133
- Add new NIRCam anomalies by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1136
- Add Logging View by mfixstsci in https://github.com/spacetelescope/jwql/pull/1080
- Remove bokeh templating from Dark monitor by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1153
- Fix API view for filenames per proposal and preview images by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1046
- Create archive page asynchronously by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1203
- Adding sort by category by mfixstsci in https://github.com/spacetelescope/jwql/pull/1155
- Improve bad pixel monitor plots by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1202
- List viewed/new status by instrument by melanieclarke in https://github.com/spacetelescope/jwql/pull/1197
- Display all detectors by melanieclarke in https://github.com/spacetelescope/jwql/pull/1213
- Remove bokeh templating from Bias Monitor by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1230
- Reformat query page functionality by BradleySappington in https://github.com/spacetelescope/jwql/pull/1240
- Add download links to image pages by melanieclarke in https://github.com/spacetelescope/jwql/pull/1283
- Improve explore image by melanieclarke in https://github.com/spacetelescope/jwql/pull/1282
- Remove Bokeh Templating from Readnoise Monitor by mfixstsci in https://github.com/spacetelescope/jwql/pull/1238
- Move Date Range selection to Query Page by BradleySappington in https://github.com/spacetelescope/jwql/pull/1396

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Make Readnoise Monitor less memory-intensive by bsunnquist in https://github.com/spacetelescope/jwql/pull/922
- Speed up MAST queries by bhilbert4 in https://github.com/spacetelescope/jwql/pull/942
- Lock python by BradleySappington in https://github.com/spacetelescope/jwql/pull/955
- Reduce memory usage by the dark monitor by bhilbert4 in https://github.com/spacetelescope/jwql/pull/908
- Fix clipping logic in preview image scaling by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1011
- Cosmic ray monitor (via mengesser:develop) by bourque in https://github.com/spacetelescope/jwql/pull/700
- nirspec TA monitors: wata and msata by penaguerrero in https://github.com/spacetelescope/jwql/pull/888
- allow per-instrument database table reset by york-stsci in https://github.com/spacetelescope/jwql/pull/1031
- Celery monitors by york-stsci in https://github.com/spacetelescope/jwql/pull/921
- Raise the upper limit on number of returned Mast entries by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1048
- Added chunked iteration to CR monitor by york-stsci in https://github.com/spacetelescope/jwql/pull/1066
- Created a clone_tables script by york-stsci in https://github.com/spacetelescope/jwql/pull/1059
- Add EDB telemetry monitor by bhilbert4 in https://github.com/spacetelescope/jwql/pull/832
- Tweaks to mnemonics queried by the EDB monitor by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1115
- Fix failing MIRI dark montior by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1129
- Unit test cleanup by melanieclarke in https://github.com/spacetelescope/jwql/pull/1167
- Catch missing obs in navigation data by melanieclarke in https://github.com/spacetelescope/jwql/pull/1206
- Update preview image generator to handle new NaN pixels by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1212
- drop support for Python 3.8 by zacharyburnett in https://github.com/spacetelescope/jwql/pull/1249
- Adding Python 3.10 Support to Testing Matrix by mfixstsci in https://github.com/spacetelescope/jwql/pull/1257
- Remove mast monitor by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1256
- Improve general JWQL queries by melanieclarke in https://github.com/spacetelescope/jwql/pull/1281
- Prepare models for external postgres db by BradleySappington in https://github.com/spacetelescope/jwql/pull/1287
- Pipeline skip already run steps by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1262
- Include migrations in source control by BradleySappington in https://github.com/spacetelescope/jwql/pull/1364
- Add step_args to calwebb_detector1_save_jump by bhilbert4 in https://github.com/spacetelescope/jwql/pull/1358

--Full Changelog--: https://github.com/spacetelescope/jwql/compare/1.1.0...1.2.0

Unreleased
==========

Web Application
~~~~~~~~~~~~~~~

- Dark and Bad Pixel Monitors delete unnecessary outputs to save disk space (933)

==================

- Correction of bad wheel file in version 1.1.0

===================

New Features
------------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Adding RTD Requirements file and yml
- Updated JDOX links so they don't produce 404 errors
- Updates to About page


Web Application
~~~~~~~~~~~~~~~

- Speed up Webapp loading
- Stripped code for creating mosaic preview images
- Ignoring unuseful pipeline products
- Updated feature desciption on EDB page
- Closed all open sessions to db explicitly
- Fix/Modify Integration Number slider
- Switch filesystem and astroquery.mast to point to OPS
- Updates to BANDIT CI
- Made plotting limits robust to missing data
- Removing reliance on jwst-dms-edb
- Updated MIRI Readout Patterns
- Updated "Filter By" Dropdown in Archive View
- Plot non-numerical mnemonic dataset


``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Removed python 3.7 support
- Added python 3.9 environment
- Updated README to reflect changes in environment files.

===================

New Features
------------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Included documentation describing the algorithms used for instrument monitors
- Included templates for bug reports and feature requests as well as a link to submit issues on the ``home`` page
- Included ``Documentation`` option on instrument drop down menu in the navigation bar


Web Application
~~~~~~~~~~~~~~~

- All relevant instruments (NIRCam, NIRISS, NIRSpec) are incorporated into the Bias Monitor
- The generalized pipeline function from ``pipeline_tools.py`` is used for image calibration
- For exposures containing multiple integrations, created thumbnail image for only a single ingtegration
- Implemented tests for all Monitor webpages


``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Modified filesystem to reflect MAST-like organizational structure
- Implemented application security tools into GitHub Actions builds
- Switched filesystem to use latest DMS build
- Improved configuration file handling


Bug Fixes
---------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Include API Docs for all modules, classes, and functions in ReadTheDocs


Web Application
~~~~~~~~~~~~~~~

- Included various fixes and improvements suggested during stakeholder demonstrations
- Fixed ``View Image``, ``View Proposal``, and ``View Header`` buttons
- Updated dashboard such that ``NULL`` Monitor Status values were fixed
- Updated options that users can select for general image queries
- Eliminated pop-up warning when viewing fits file headers in the web app
- Removed login button on homepage and supporting authentication code and replaced with SSO authentication


``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Fixed Dark Monitor failure due to dither point keyword values in data
- Addressed Dark Monitor failure with data including 64-bit integers
- Resolved the NIRSpec amp boundary issue
- Fixed missing reference file error handling with latest DMS build in Readnoise Monitor
- Ensured Dark, Bias, and Readnoise Monitors complete successfully with latest DMS build
- Included read persmissions for database tables
- Performed flake8 sweep of the code repository
- Updated software to support the latest versions of ``bokeh`` and ``pysiaf``
- Removed dependencies on the astroconda channel

===================

New Features
------------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Included badges for GitHub Actions in ``README.md``
- Replaced Jenkins support with GitHub Actions for CI testing
- Removed Python 3.6 support


Web Application
~~~~~~~~~~~~~~~

- Allow local developers to bypass ``auth.mast`` authentication
- Added cosmic ray monitor, unit test code, and relevant database files
- Removed unsupported options including image anomalies, unlooked images and monitors that haven't been implemented
- Improved loading times for web app archive pages by generating a representative thumbnail with ``generate_proposal_thumbnail.py``, using ``astroquery.Mast`` rather than scraping filesystem, adding optional checks that file paths exist
- Implemented JWQL Dashboard View

``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Updated the call to the JWST pipeline RSCD step
- Added GitHub Actions


Bug Fixes
---------

Web Application
~~~~~~~~~~~~~~~

- Improved loading times of readnoise monitor webpage
- Fixed broken bokeh CDN links

``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Dark monitor exits gracefully when there are not enough files in the filesystem
- Fixed filename parsing for WFS&C and AMI products
- Adjusted generation of preview images such that images are created for all file types
- Update ops naming convention in log directory

===================

New Features
------------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Added project citation information to ``README``, along with a Zenodo badge
- Added API Documentation for ``bokeh`` templating software

Web Application
~~~~~~~~~~~~~~~

- Reorganized and made aesthetic improvements to instrument landing pages to be more user-friendly
- Enabled more dynamic search options and aesthetic improvements for anomaly query webpage
- Added web app view for displaying a particular table of the ``jwqldb`` database
- Added webpage for displaying Bias Monitor results with ``bokeh`` plots

``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Changed ``utils.credentials.py`` to always authenticate a MAST user with a user-identified token in the ``config.json`` file, instead of using a cached token, which was sometimes causes errors
- Updated software to support the latest versions of ``django`` and ``bokeh``
- Removed ``affected_tables`` column of ``monitor`` database table, as it stored redundant information
- Updated the Readnoise Monitor to work for all JWST instruments


Bug Fixes
---------


Web Application
~~~~~~~~~~~~~~~

- Fixed bug in Readnoise Monitor webpage that was causing the web app and ``jwql`` database to hang

``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Fixed bug that was causing the ``test_amplifier_info()`` test in ``test_instrument_properties.py`` to fail; truth values were updated to reflect a change in the format of the returned dictionaries from the ``amplifier_info()`` function
- Fixed bug in ``get_header_info()`` that was causing ``test_data_containers.py`` to fail; the function expected the filename without the FITS extension, and the returned header info is in a dictionary (not a string)
- Fixed bug in ``test_utils.py``, and changed ``utils.py`` to make it robust in matching upper and lowercase detector names
- Updated ``utils.instrument_properties`` fix MIRI amplifier bounds calculation when omitting reference pixels