Etlhelper

Latest version: v1.0.1

Safety actively analyzes 682471 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 3 of 5

0.11.0

Not secure
This release adds the following features:

* The `on_error` parameter is added to `executemany` and the functions that call it (`load`, `copy_rows`, `copy_table_rows`). This allows a job to complete even if some rows fail (e.g. due to failed unique constraint). A user-provided function is called on the failed rows from each chunk of data once the chunk is complete. See [README](https://github.com/BritishGeologicalSurvey/etlhelper#handling-insert-errors) for details.
* The `chunk_size` is now a named parameter in all etl functions. Previously it was set by the global variable, `etlhelper.etl.CHUNKSIZE`. This makes it more accessible.

What's Changed
* Add Python 3.9 to CI checks by volcan01010 in https://github.com/BritishGeologicalSurvey/etlhelper/pull/116
* Add on_error handling to executemany and related methods by ximenesuk and volcan01010 in https://github.com/BritishGeologicalSurvey/etlhelper/pull/115


**Full Changelog**: https://github.com/BritishGeologicalSurvey/etlhelper/compare/v0.10.0...v0.11.0

0.10.0

Not secure
This release adds two new functions that make life easier by using autogenerated INSERT statements based on dictionary keys or namedtuple attributes.

+ `load` inserts data into the given table, generating the INSERT statement based on the input data. There is a recipe for [transforming and loading](https://github.com/BritishGeologicalSurvey/etlhelper#csv-load-script-template) data from a CSV file. `load` can also combined with [`iter_rows`](https://github.com/BritishGeologicalSurvey/etlhelper#combining-iter_rows-with-load) to insert the results of arbitrary SQL queries on the source connection into the destination connection.
+ `copy_table_rows` [copies the contents of a table](https://github.com/BritishGeologicalSurvey/etlhelper#copy-table-rows) from one connection to another.

What's Changed
* Update spatial ETL GeomFromText command by volcan01010 in https://github.com/BritishGeologicalSurvey/etlhelper/pull/106
* Load function by volcan01010 in https://github.com/BritishGeologicalSurvey/etlhelper/pull/109


**Full Changelog**: https://github.com/BritishGeologicalSurvey/etlhelper/compare/v0.9.7...v0.10.0

0.9.7

Not secure
This minor release includes:

+ Recipe for [database to API transfer](https://github.com/BritishGeologicalSurvey/etlhelper#database-to-api--nosql-copy-etl-script-template) uses asynchronous `aiohttp` for ~10x speedup.
+ `iter_chunks`, `fetchmany`, `fetchone` close transactions when they finish.
+ README describes using `fetchone` with [queries using `SELECT ... RETURNING`](https://github.com/BritishGeologicalSurvey/etlhelper#insert-rows) syntax.

0.9.6

Not secure
This minor release includes:

+ Use `fast_executemany` by default for MS SQL Server (via PyODBC driver). This provided a 100x increase in insert speed during our tests. See [README.md](https://github.com/BritishGeologicalSurvey/etlhelper#disabling-fast_executemany-for-sql-server-and-other-pyodbc-connections) for details.

0.9.5

Not secure
This minor release includes:

+ allow setting of `row_factory` in `copy_rows`. Thanks to Real-Gecko.
+ basic database to API ETL pipeline script added as recipe in README.md

0.9.4

Not secure
This minor release includes:

+ `setup_oracle_client` fails gracefully with sys.exit() instead of producing stack trace
+ README highlights `iter_rows`
+ Fix typos in README example ETL script

Page 3 of 5

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.