Automunge

- some additional quality control conducted
- performed a visual inspection on entire inversion library
- in the process identified a few small snafus
- bnep/bneo/tlbn -> found and fixed inversion edge case associated with all non-numeric data in train set
- splt/sbst -> found and fixed inversion bug originating from new suffix parameter support from 6.19
- in the process developed a new validation test for inversion library to be performed prior to rollouts
- which should help catch edge cases going forward for inversion

- sort of a quality control audit / additional walkthrough of edits from last week
- found and fixed bug in pwr2 inversion function originating from new suffix convention from 6.19
- everything else looked good
- a few code comments added here and there
- removed the '\' character in family tree definitions since shown in read me
- and since went that far went ahead and conformed the process_dict intitializer to match

- ok just realized that the column_map had an edge case
- associated with the excl transform
- where excl is direct pass-through, and is unique in library for suffix convention
- in that suffix is recorded in internal data structures to support operation
- and then removed in the returned dataframe so that returned column title is consistent with recieved
- which makes sense since excl is for use on direct pass-through with no infill
- (where if excl suffix retention is desired in returned data to support data structure navigation the excl_suffix parameter can still be activated)
- anyway point was that the column_map for excl columns was showing the internal representation with suffix even when returned data did not include
- so new convention is that excl column suffix convention in column_map is consistent with returned data

- new convention for the library
- now every transform supports the 'suffix' parameter
- which can be passed as a (traditionally 4 character) string
- which will be appended on the returned column(s) with a leading underscore
- previously this parameter was only supported in a selection of transforms
- where without the parameter the suffix was a hard coded property of a transformation function
- the benefit of the convention is to support use case where user wishes to redundantly perform the same transform on the same input column but with different parameters
- as an example to conduct redundant bin aggregators with different boundaries
- the only requirement to facilitate is that each of the redundant applications have the transformation function associated with a distinct transformation category and a distinct suffix parameter entry
- where the suffix parameter could be part of the category's processdict definition by use of a defaultparams entry
- such that each of the distinct transformation categories can be entered into the root category family tree
- only exception to this convention is for the text and excl transforms which have some quirks with suffix conventions
- updated the pwrs transform suffix convention to support suffix parameter
- fixed validation function printout associated with new recorded_category processdict entry introduced in 6.16
- found and fixed a small formatting snafu in splt and sbst transforms associated with int_headers option

- added inplace parameter to processing function normalization dictionaries
- which ommission was kind of a shortcut in the first place, better to have it accessible
- standardized on single convention for suffix parameter regarding inclusion of underscore
- such that transforms that accept a suffix parameter should pass just the (traditionally 4 character) string and a leading underscore will be added internally
- standardized on single form of postprocess functions with respect to accessing normkey
- in the process found a channel for error previously missed
- for cases when a transform returns an empty set and was selected for inplace
- as without a returned set the inplace operation doesn't replace the input column
- resolved by standardizing on inspection of inplace param in empty set scenario
- which means we will need to retain param inspections in postmunge after all
- no big deal is not a significant amount of overhead
- also found and fixed a small edge case for Binary dimensionality reduction when applied to an all numeric set (which is a pass through)
- a few small cleanups and code comments for support function _postprocess_textsupport
- update to the demonstration of custom postprocess function definition in read me

- ok last rollout was a great step in right direction
- but still fell a little short of fully standardized method to extract a normkey in postprocess functions
- realized that by revising the columnkey_dict population from basing on recorded category
- to basing on transformation category (as populated in family tree)
- then had ability to eliminate need to inspect any transformation parameters in postprocess transforms
- which were previously compared to ensure in cases of redundant transformations to same inputcolumn we were accessing version with right parameters
- now since cases of redundant transfomations to same input column won't be populated in the same categorylist_aggregate entry to a column_key_dict entry associated with a transformation category
- we've eliminated the scenario requiring inspection of parameters
- note that we already had the convention that a transformation category can only be entered once in a set of upstream or downstream parameters (although may be entered in both if desired)
- which is validated in the _check_transformdict2 support function
- this update is a great step in standardizing on form of postprocessing functions
- updated to new convention of accessing normkey in postprocess transformations for onht/text/splt/sp19/sbst/sbs3/hash/pwrs/bnwd/bnep/tlbn/bkt1/bkt2/smth/spl2/srch/src2/src3
- we intend in future updates to take advantage of this to fully standardize on postprocessing transform normkey retrieval accross the entire library
- and also to eliminate inspection of transformation parameters in postmunge which will clear up a little overhead to speed things up
- also small cleanup struck a postmunge columnkey variable initialization no longer needed after 6.16