Pyhum

1. Fixed small bug in e1e2 module that was using smoothed easting/northing coordinates rather than actual coordinates
2. in map module, if mode =3 (nearest neighbour with gaussina smoothing), the sigma on the gaussian function has been changed from 1 to 0.1, due to sigma = 1 oversmoothing. Will implement this as a user input variable in a future release
3. the 'correct' module now has a new flag, 'correct_withwater'. If correct_withwater=0 [default], the usual/old behaviour is observed. If correct_withwater=1, radiometric corrections are carried out on scans without water column removed, and a new set of files (*__data_star_lw.dat and *_data_port_lw.dat) are produced


v.1.3.2
1) map and map_texture:
implemented various ways to stop the programs crashing if low on memory, due to a grid size creating gridded data sets too large for system memory. If the system is about to crash, resolutions are doubled and functions rethrown. It will continue to do this until a gridded data set has been created.

Gridding routines now rely on the fast pyresample module. 3 types of nearest neighbour interpolation are now supported: 1) ordinary; 2) inverse distance weighted; and 3) gaussian weighted. Use the 'mode' flag in 'map' and 'map_texture'

2) Point cloud outputs from the “map” module now contain georeferenced depths, ranges and headings as well as sidescan pixel intensities. This is useful in its own right, for subsequent analysis, but mostly in anticipation of a forthcoming new module called 'mosaic' which will attempt to mosaic adjacent scans together and will utilise this information

3) the program will now exit at the start of 'read' if the epsg code is not compatible with Basemap (which will cause problems later in the map and map_texture modules)

4) the 'correct' module optionally now applies a phase preserving filter to the data (via the 'dofilt' flag). Use with caution: it can be very slow for very large and very noisy data sets. By default, it is turned off.

5) the 'read' module now deals with data that get corrupted or lost when the user hits the 'stop recording' button before the latest ping has been written to file (this rare occasion can cause unequal numbers of pings in starboard and port, or scans). Also some changes have been made to the automatic bed picking algorithm which hopefully help that process (but it remains imperfect – always check and revert to 'manual' bedpick=2 if the bed pick is bad)

6) the rmshadows module has changed: in 'automatic mode' the user no longer specifies the 'number of k-means'. Instead shadow removal is achieved by examining the grey-level co-occurrence matrix of sidescan textures and removes areas of low texture. Seems like a better approach but more work/tests are necessary. Please report any feedback on this module. The 'manual' mode remains unchanged

7) read the module now utilizes IDX files to help speed up the process of reading data from the SON files. IDX files contain indices of start and stop pings. If these files are missing, it's not crucial – the program will revert to the old method and will take slightly longer for large files

1) general
a) underscores, negatives and spaces are now removed from variable 'base'

b) all modules protected from parallel processing routines on Windows by wrapping with “if **name** == '**main**':”

c) all modules now include default values in the “def” declaration

d) file path construction using os.path.normpath with os.path.join is more reliable on Windows systems than concatenating strings of file paths and parts. Thanks to Dan Hamill for this.

e) all memory mapped files now use file pointers rather than filenames in reading and writing, which is safer way to access memory mapped files on Windows

2) _pyread.pyx:getmetadata
remove headings when gps flag 1 (gps1) is zero and when gps flag 2 (gps2) is zero, and use humutils.nan_helper to interpolate over

3) _pyhum_map:make_map (and _pyhum_map_texture)
a) when masking the gridded data, there was a bug at lower grid resolutions which caused masking of the entire grid. Fixed with the clause, if np.floor(np.sqrt(1/res))-1, distances greater than np.floor(np.sqrt(1/res))-1 are masked, otherwise, distances greater than np.sqrt(1/res) are masked

b) images loaded into kml files are better resolved with smaller bounding box

c) faster gridding routine using kd-trees rather than scipy's griddata

4) _pyhum_e1e2:e1e2
when filling nans in variables rough, hard, sv_e1 and sv_e2, this now has to be put in a try/except/continue loop to stop the program crashing occasionally

5) _pyhum_correct:correct
uses parallel processing to do radiometric corrections, for greater speed

6) New module: _pyhum_rmshadows, which is designed to remove acoustic shadows caused by shallows, shorelines and other large obstacles on the edges of scans. Shadow removal can either be either manual (user is prompted to delineate the shadows with the cursor) or automatic (image is posterized using a kmeans algorithm, with a user-specified number of means, then the portion of the image associated with the lowest mean intensity is removed)