Sdss-marvin

--------------------

Added
^^^^^

- Added new query_params object, for easier navigation of available
query parameters. Added new tests.
- Added a new guided query builder using Jquery Query Builder to the
Search page
- Added a View Galaxies link on the web results to view postage stamps
of the galaxies in the results
- Added Route Rate Limiting. Adopts a limit of 200/min for all api
routes and 60/minute for query api calls and web searches

Changed
^^^^^^^

- Changed call signature for
:meth:``marvin.utils.plot.map.no_coverage_mask`` (removed ``value``
arg because unused, added ``None`` as default value ``ivar``
(``None``), and re-ordered args and kwargs).
- Changed call signature for
:meth:``marvin.utils.plot.map.bad_data_mask`` (removed ``value`` arg
because unused).
- Changed the Marvin web search page to use the new query_params and
parameter grouping. Removed the autocomplete input box.
- Updated the documentation on query and query_params.
- Modified Guided Search operator options to remove options that could
not be parsed by SQLA boolean_search
- Refactored the web settings, route registration, extensions to enable
extensibility
- Issue `282 <https://github.com/sdss/marvin/issues/282>`_: Improvements to "Go to CAS" link. Changed to Go To
SkyServer and updated link to public up-to-date link

Fixed
^^^^^

- Issue `102 <https://github.com/sdss/marvin/issues/102>`_: problem with urllib package when attempting to retrieve
the Marvin URLMap
- Issue `93 <https://github.com/sdss/marvin/issues/93>`_: safari browser does not play well with marvin
- Issue `155 <https://github.com/sdss/marvin/issues/155>`_: Contrails in Web Map
- Issue `174 <https://github.com/sdss/marvin/issues/174>`_: sdss_access may not be completely python 3 compatible
- Issue `196 <https://github.com/sdss/marvin/issues/196>`_: Bin not loading from local sas
- Issue `207 <https://github.com/sdss/marvin/issues/207>`_: Get Maps in MapSpecView of Galaxy page sometimes fails to
return selected maps
- Issue `210 <https://github.com/sdss/marvin/issues/210>`_: pip upgrade may not install new things as fresh install
- Issue `209 <https://github.com/sdss/marvin/issues/209>`_: marvin version from pip install is incorrect
- Issue `268 <https://github.com/sdss/marvin/issues/268>`_: Cube flux from file error
- Issue `85 <https://github.com/sdss/marvin/issues/85>`_: Python does not start in Python 3
- Issue `273 <https://github.com/sdss/marvin/issues/273>`_: ha.value bug
- Issue `277 <https://github.com/sdss/marvin/issues/277>`_: Ticks for log normalized colorbar
- Issue `275 <https://github.com/sdss/marvin/issues/275>`_: logger crashes on warning when other loggers try to log
- Issue `258 <https://github.com/sdss/marvin/issues/258>`_: 422 Invalid Parameters
- Issue `271 <https://github.com/sdss/marvin/issues/271>`_: Problem in dowloading image.
- Issue `97 <https://github.com/sdss/marvin/issues/97>`_: sqlalchemy-boolean-search not found when installed from
pip source
- Issue `227 <https://github.com/sdss/marvin/issues/227>`_: Marvin installation in python 3.6 (update setuptools to
36)
- Issue `262 <https://github.com/sdss/marvin/issues/262>`_: problem with marvin update
- Issue `270 <https://github.com/sdss/marvin/issues/270>`_: BPT array sizing not compatible
- Issue `88 <https://github.com/sdss/marvin/issues/88>`_: Deployment at Utah requires automatisation
- Issue `234 <https://github.com/sdss/marvin/issues/234>`_: Add (and use) functions to the datamodel to determine
plotting parameters
- Issue `278 <https://github.com/sdss/marvin/issues/278>`_: marvin_test_if decorator breaks in python 2.7
- Issue `274 <https://github.com/sdss/marvin/issues/274>`_: cube slicing to get a spaxel fails with maps error
- Issue `39 <https://github.com/sdss/marvin/issues/39>`_: implement more complete testing framework
- Issue `242 <https://github.com/sdss/marvin/issues/242>`_: Result object representation error with 0 query results
- Issue `159 <https://github.com/sdss/marvin/issues/159>`_: Marvin issues multiple warnings in PY3
- Issue `149 <https://github.com/sdss/marvin/issues/149>`_: Improve integrated flux maps display in web

--------------------

Added
^^^^^

- Issue `204 <https://github.com/sdss/marvin/issues/204>`_: added elpetro_absmag colours to mangaSampleDB models.
- Issue `253 <https://github.com/sdss/marvin/issues/253>`_: Plotting tutorial.
- Issue `223 <https://github.com/sdss/marvin/issues/223>`_: Easy multi-panel map plotting (with correctly placed
colorbars).
- Issue 232 and Issue `251 <https://github.com/sdss/marvin/issues/251>`_: Uses matplotlib style sheets context
managers for plotting (map, spectrum, and BPT) and restores previous
defaults before methods finish.
- Issue `189 <https://github.com/sdss/marvin/issues/189>`_: Map plotting accepts user-defined value, ivar, and/or
mask (including BPT masks).
- Issue `252 <https://github.com/sdss/marvin/issues/252>`_: Quantile clipping for properties other than velocity,
sigma, or flux in web.
- Added ``utils.plot.map`` doc page.
- Added ``tools.map`` doc page.

Changed
^^^^^^^

- Issue `243 <https://github.com/sdss/marvin/issues/243>`_: inverted ``__getitem__`` behaviour for
Cube/Maps/ModelCube and fixed tests.
- Modified Flask Profiler File to always point to
$MARVIN_DIR/flask_profiler.sql
- Issue `241 <https://github.com/sdss/marvin/issues/241>`_: Moved map plotting methods from tools/map to
utils/plot/map
- Issue 229 and Issue `231 <https://github.com/sdss/marvin/issues/231>`_: Switch to new gray/hatching scheme (in
tools and web):

- gray: spaxels with NOCOV.
- hatched: spaxels with bad data (UNRELIABLE and DONOTUSE) or S/N
below some minimum value.
- colored: good data.

- Issue `238 <https://github.com/sdss/marvin/issues/238>`_: Move plot defaults to datamodel (i.e., bitmasks,
colormaps, percentile clips, symmetric, minimum SNR).
- Issue `206 <https://github.com/sdss/marvin/issues/206>`_: SNR minimum to None (effectively 0) for velocity maps so
that they aren't hatched near the zero velocity contour.
- Simplified default colormap name to "linearlab."
- Decreased map plot title font size in web so that it does not run
onto second line and overlap plot.

Fixed
^^^^^

- Interactive prompt for username in sdss_access now works for Python
3.
- Fixed `195 <https://github.com/sdss/marvin/issues/195>`_: The data file for the default colormap for ``Map.plot()``
("linear_Lab") is now included in pip version of Marvin and does not
throw invalid ``FileNotFoundError`` if the data file is missing.
- Fixed `143 <https://github.com/sdss/marvin/issues/143>`_: prevents access mode to go in to remote if filename is
present.
- Fixed `213 <https://github.com/sdss/marvin/issues/213>`_: shortcuts are now only applied on full words, to avoid
blind replacements.
- Fixed `206 <https://github.com/sdss/marvin/issues/206>`_: no longer masks spaxels close to zero velocity contour in
web and tools map plots
- Fixed `229 <https://github.com/sdss/marvin/issues/229>`_: corrects web bitmask parsing for map plots
- Fixed `231 <https://github.com/sdss/marvin/issues/231>`_: hatch regions within IFU but without data in map plots
- Fixed `255 <https://github.com/sdss/marvin/issues/255>`_: Lean tutorial code cells did not work with the ipython
directive, so they now use the python directive.
- Highcharts draggable legend cdn.

Removed
^^^^^^^

- Issue 232 and Issue `251 <https://github.com/sdss/marvin/issues/251>`_: Automatic setting of matplotlib style
sheets via seaborn import or ``plt.style.use()``.

--------------------

Added
^^^^^

- API and Web argument validation using webargs and marshmallow. If
parameters invalid, returns 422 status.

Changed
^^^^^^^

- Per Issue `186 <https://github.com/sdss/marvin/issues/186>`_: Switched to using the elpetro version of stellar
mass, absolute magnitude i-band, and i-band mass-to-light ratio for
NSA web display, from sersic values. (elpetro_logmass,
elpetro_absmag_i, elpetro_mtol_i)
- Issue `188 <https://github.com/sdss/marvin/issues/188>`_: deprecated snr in favour of snr_min for get_bpt. snr can
still be used.
- Issue `187 <https://github.com/sdss/marvin/issues/187>`_: Renamed NSA Display tab in web to Galaxy Properties.
Added a link to the NASA-Sloan Atlas catalogue to the table title.
- Moved our documentation to readthedocs for version control. Updated
all Marvin web documenation links to point to readthedocs.

Fixed
^^^^^

- A bug in the calculation of the composite mask for BPT.
- Issue `179 <https://github.com/sdss/marvin/issues/179>`_: Fixed a python 2/3 exception error compatibility with the
2.1 release.

--------------------

Added
^^^^^

- Added query runtime output in search page html. And a warning if
query is larger than 20 seconds.

Changed
^^^^^^^

- Removed the python 3 raise Exception in the check_marvin bin
- Reverted the api/query return output from jsonify back to json.dumps

- This is an issue with python 2.7.3 namedtuple vs 2.7.11+

Fixed
^^^^^

- Issue `181 <https://github.com/sdss/marvin/issues/181>`_: web display of maps were inverted; changed to xyz[jj, ii,
val] in heatmap.js
- Added more code to handle MarvinSentry exceptions to fix 179.

--------------------

Added
^^^^^

- Restructured documentation index page.
- Improved installation documentation:

- Removed old installation text
- Added section on marvin SDSS dependencies and SAS_BASE_DIR
- Added section for FAQ about installation
- Added web browser cache issue into FAQ

- Added traceback info in the API calls

- Added traceback attribute in Brain config
- Added hidden \_traceback attribute in Marvin config
- Only implemented in two Query API calls at the moment
- Added a few tests for traceback
- see usage in cube_query in marvin/api/query.py

- Added the Ha_to_Hb ratio the DAP ModelClasses for querying
- Added new script to perform somce basic system, os, and Marvin
checks: bin/check_marvin
- Added an alert banner when the user is using Safari. See 94.
- Issue `122 <https://github.com/sdss/marvin/issues/122>`_: added ra/dec to spaxel
- Issue `145 <https://github.com/sdss/marvin/issues/145>`_: Limited the number of query parameters in the web
- Added more tests to Results for sorting, paging, and getting subsets
- Added kwargs input for Spaxel when using Result.convertToTool
- Added automatic Sentry error logging 147 into MarvinError, and
Sentry in Flask for production mode
- Added custom error handlers for the web, with potential user feedback
form
- Added Sentry tool for grabbing and displaying Sentry statistics
- Added text to MarvinError with a Github Issues link and description
of how to submit and issue
- Added Results option to save to CSV
- Added new parameters in Marvin Config to turn off Sentry error
handling and Github Issue message
- Added Python example code for getting a spectrum in galaxy page of
web.
- Added new test for image utilities getRandomImages, getImagesByPlate,
getImagesByList
- Added new documentation on Image Utilities
- Added new image utility function showImage, which displays images
from your local SAS
- Added the Kewley+06 implementation of the BPT classification as
``Maps.get_bpt()``
- Added quick access to the NSA information for a Cube/Maps either from
mangaSampleDB or drpall.

Changed
^^^^^^^

- When marvin is running from source (not dist), ``marvin.__version__``
is ``dev``.
- Removed the cleanUpQueries method to assess db stability
- Switched dogpile.cache from using a file to python-memcached
- Syntax changes and bug fixes to get Marvin Web working when Marvin
run on 3.5
- Got Queries and Results working in 3.5
- Changed all convertToTool options in Results from mangaid to plateifu
- Added release explicitly into api query routes
- Modified the decision tree in query to throw an error in local mode
- Modified convertToTool to accept a mode keyword
- Modifed the MarvinError for optional Sentry exception catching, and
github issue inclusion
- Updated all Marvin tests to turn off Sentry exception catching and
the github message
- Updated some of the Tools Snippets on the web
- Overhauled Map plotting

- uses DAP bitmasks (NOVALUE, BADVALUE, MATHERROR, BADFIT, and
DONOTUSE)
- adds percentile and sigma clipping
- adds hatching for regions with data (i.e., a spectrum) but no
measurement by the DAP
- adds Linear Lab color map
- adds option for logarithmic colorbar
- adds option to use sky coordinates
- adds map property name as title
- makes plot square
- sets plotting defaults:

- cmap is linear_Lab (sequential)
- cmap is RdBu_r (diverging) for velocity plots (Note: this is
reversed from the sense of the default coolwarm colormap in
v2.0---red for positive velocities and blue for negative
velocities)
- cmap is inferno (sequential) for sigma plots
- clips at 5th and 95th percentiles
- clips at 10th and 90th percentiles for velocity and sigma plots
- velocity plots are symmetric about 0
- uses DAP bitmasks NOVALUE, BADVALUE, MATHERROR, BADFIT, and
DONOTUSE
- also masks spaxels with ivar=0
- minimum SNR is 1

- Changed Marvin Plate path back to the standard MarvinToolsClass use
- Made sdss_access somewhat more Python 3 compatible
- Modified the image utilities to return local paths in local/remote
modes and url paths when as_url is True
- downloadList utility function now downloads images
- updated the limit-as parameter in the uwsgi ini file to 4096 mb from
1024 mb for production environment

Fixed
^^^^^

- Issue `115 <https://github.com/sdss/marvin/issues/115>`_: drpall does not get updated when a tool sets a custom
release.
- Issue `107 <https://github.com/sdss/marvin/issues/107>`_: missing os library under save function of Map class
- Issue `117 <https://github.com/sdss/marvin/issues/117>`_: hybrid colours were incorrect as they were being derived
from petroth50_el.
- Issue `119 <https://github.com/sdss/marvin/issues/119>`_: test_get_spaxel_no_db fails
- Issue `121 <https://github.com/sdss/marvin/issues/121>`_: bugfix with misspelled word in downloadList utility
function
- Issue `105 <https://github.com/sdss/marvin/issues/105>`_: query results convertToTool not robust when null/default
parameters not present
- Issue `136 <https://github.com/sdss/marvin/issues/136>`_: BinTest errors when nose2 run in py3.5 and marvin server
in 3.5
- Issue `137 <https://github.com/sdss/marvin/issues/137>`_: PIL should work in py2.7 and py3.5
- Issue `172 <https://github.com/sdss/marvin/issues/172>`_: broken mode=auto in image utilities
- Issue `158 <https://github.com/sdss/marvin/issues/158>`_: version discrepancy in setup.py

--------------------

Added
^^^^^

- Docs now use ``marvin.__version__``.

Fixed
^^^^^

- Fixed 100, `103 <https://github.com/sdss/marvin/issues/103>`_: problem with getMap for properties without ivar.
- Fixed `101 <https://github.com/sdss/marvin/issues/101>`_: problem with marvin query.