Aws-insurancelake-etl

Documentation updates

New v3.3 includes:
- Using SQL documentation
- Documentation on using data freshness checks to manage workflow dependencies (part of How to Manage Data Quality)
- FIX: Overwrite cleanse bucket partitions more effectively
- FIX: After-SparkSQL quarantine rules properly quarantine rows

New v3.2 release includes:
- JSON format support and structured data transforms
- Schema mapping documentation
- File format documentation
- Other documentation and code improvements
- FIX: Consistent use of job_last_updated field in job audit table
- FIX: Data security transforms halt on missing field
- FIX: Schema mapping recommendation file has a header
- FIX: Clean columns supports removing carriage returns

New v3.1 includes:
- Improved performance of filldown transform
- Introduced rownumber transform
- Improved lookup transform to support global lookups
- Improved currency transform to handle non-string columns
- Improved etl_cleanup script to support more options
- FIX: Documentation of changetype transform

New version includes:
- New Documentation: Transform reference, Data Quality reference
- Glue Data Quality available at 3 stages of the data pipeline
- New transforms: columnreplace, multiplycolumns, filldown, changetype
- Codestar connection support
- Entity matching and Hudi table support
- FIX: CDK-nag Python 3.12 error
- FIX: Glue Catalog integration handling of schema evolution