Tradedangerous

Fix

* fix: spansh- fix overzealous download

spansh plugin now will not download the source file if the source has not been modified more recently than the local copy downloaded previously
if the source file has been modified more recently, it is downloaded, overwriting the local copy, if any ([`5a8dd2a`](https://github.com/eyeonus/Trade-Dangerous/commit/5a8dd2a3676c48fd95c73ca1dbc72e098428960d))

Fix

* fix: can&39;t change if doesn&39;t exist yet ([`59512b0`](https://github.com/eyeonus/Trade-Dangerous/commit/59512b0a61565bbe3164a4cfbfc72983b07a20cb))

Chore

* chore: fix Path error ([`5f5db6e`](https://github.com/eyeonus/Trade-Dangerous/commit/5f5db6e05ed911ea89cbb51ac8ca0dadf9c1572c))

Feature

* feat: include more tables in spansh import

Added processing spansh source for Ship, ShipVendor, Upgrade, and UpgradeVendor tables. ([`da1ee03`](https://github.com/eyeonus/Trade-Dangerous/commit/da1ee0324744294f8f26e7ddef2f4dfa8d9352c8))

Refactor

* refactor: correct `get_timings()` return hint ([`beeaea7`](https://github.com/eyeonus/Trade-Dangerous/commit/beeaea7fd0cb3bbb39b910afc3535cae3617220f))

* refactor: accurate progress bar ([`293c35b`](https://github.com/eyeonus/Trade-Dangerous/commit/293c35b34631767afce43325c018f106416c4f92))

* refactor: accurate progress bar ([`a90a216`](https://github.com/eyeonus/Trade-Dangerous/commit/a90a216ff2c6f386d7801d68a6b6ddc54e948455))

* refactor: `ingest_stream()` -> `self.ingest_stream()` ([`0f18d10`](https://github.com/eyeonus/Trade-Dangerous/commit/0f18d10e4039f63c7f38e4593e9f7eeb9df912b4))

* refactor: correct `load_known_stations` return hint ([`0eaed90`](https://github.com/eyeonus/Trade-Dangerous/commit/0eaed909f1da4238dc3417abaebd73a4b0096e5d))

* refactor: output sample system json if debug

The entire galaxy_stations.json is too big and unwieldy to look at directly, so turning on debug will output only the `Shinrarta Dezhra` system to `./tmp/shin_dez.json`, making it much easier to see the data structure, and since `Jameson Mermorial` has all the things, we can also use it to check those things against the DB. ([`3688683`](https://github.com/eyeonus/Trade-Dangerous/commit/36886837472d24f513fce6485ae94131fedb5341))

* refactor: move eddblink_plug downloads to ./tmp ([`dee11c2`](https://github.com/eyeonus/Trade-Dangerous/commit/dee11c23738347329a46db23460b9b549345ad8f))

* refactor: include system name in &34;updated&34;/&34;added&34; logs

177 ([`4e403fe`](https://github.com/eyeonus/Trade-Dangerous/commit/4e403fef24c002f0953f0894fde8fed88f2f91ea))

Style

* style: whitespace edits ([`9f6c2b6`](https://github.com/eyeonus/Trade-Dangerous/commit/9f6c2b6b1a6384ac38df4a5e376babed2e5a660b))

* style: fix white-space ([`c07eb91`](https://github.com/eyeonus/Trade-Dangerous/commit/c07eb912ba092e39d7f24c07e0b048eb9f0522dc))

Documentation

* docs: remove Merge line from changelog ([`02ac0c6`](https://github.com/eyeonus/Trade-Dangerous/commit/02ac0c6eada715c127555837fde7dd9df2213539))

Feature

* feat: add `--maxage` to `sell` command ([`ab34568`](https://github.com/eyeonus/Trade-Dangerous/commit/ab34568ba1bdd4aca5f236595f0c519d0f8e0f64))

Chore

* chore: Fix linting error about collections.abc ([`1358479`](https://github.com/eyeonus/Trade-Dangerous/commit/1358479c766bae220d6df748505506db21281c5e))

Feature

* feat: better eddblink progress reporting

- small tweaks to improve performance of eddblink import,
- commit batching to try and improve eddblink import speed,
- use rich progress bars to give better insight into commit rate

one interesting thing to try is to change the progress bar description around the COMMIT while importing prices:


prog.increment(description=&34;COMMIT&34;)
cursor.execute(&34;COMMIT&34;)
transaction_items = 0
cursor.execute(&34;BEGIN TRANSACTION&34;)
prog.increment(description=&34;Processing&34;)


and then adjust the batch size. when batchsize is too low, this makes the description flicker, but you can get a sense for how increasing the batch size reduces the total import time until the amount of memory/wal etc starts to make the commit time excessively long ([`f3af244`](https://github.com/eyeonus/Trade-Dangerous/commit/f3af244d6940a2394b4b05725c412b1f1a0796da))

* feat: database schema update

- Remove ROWID from Station table to improve performance,
- Introduce StationDemand and StationSupply tables for breaking up StationItem,
- Reduce size of station/system index, ([`32a7db1`](https://github.com/eyeonus/Trade-Dangerous/commit/32a7db15ffd041d9b47463d54c4f7e87b3f08651))

* feat: Swap out homebrew progress bars for Rich

- make bars hideable,
- polish,
- default the progress bar to visible, require show=False explicitly.
- add more bar styles,
- use some of the bars, ([`52bb7f4`](https://github.com/eyeonus/Trade-Dangerous/commit/52bb7f456990a047b0292198d40b3b29ebc0e8d9))

* feat: nicer transfer progress bars

uses the new rich-based progress bars to display download speed information etc. ([`bd4738a`](https://github.com/eyeonus/Trade-Dangerous/commit/bd4738a497ebd121fa9b3bfda42cde61a0843144))

* feat: rich progress bars

This introduces an enhancement over the old progress bars, using rich to provide colorful, live bars. Base types for skinning them are included. ([`156904b`](https://github.com/eyeonus/Trade-Dangerous/commit/156904b9eca9c3090873dba787914fae2a98a6f2))

Fix

* fix: Python version <3.9 does not support parenthesized context expressions ([`aab81db`](https://github.com/eyeonus/Trade-Dangerous/commit/aab81dba2b4364c5a5bb954602182c26927b9c31))

* fix: progress bar display/updates and linting

- advance progress bars properly,
- actually display the text of the progress bar, duh ([`ec94c0b`](https://github.com/eyeonus/Trade-Dangerous/commit/ec94c0b18efd1ea2f2324f67d01de6260e790099))

Refactor

* refactor: make transfers use the rich bars properly. ([`ca12c5f`](https://github.com/eyeonus/Trade-Dangerous/commit/ca12c5fdf18e762da59de45fa18456b6060d69df))

* refactor: move file_line_count into fs ([`8a9989b`](https://github.com/eyeonus/Trade-Dangerous/commit/8a9989be17cdd0e578d9bfff40b8a96e2850e91a))

Unknown

* - feat: add LongRunningCountBar to progress bars

Line CountingBar except it includes a TimeRemaining column (long running tasks seem to warrant this) ([`6f188a4`](https://github.com/eyeonus/Trade-Dangerous/commit/6f188a4a3c03eb5640c7fadc5aa408f731bbf5ea))

Fix

* fix: linter warnings from moving the shebang from line 1 ([`73d4a0d`](https://github.com/eyeonus/Trade-Dangerous/commit/73d4a0d25a633b6cd622459369b8ca9ef044f83a))

* fix: multiple options to --pad-size and others was no-longer working ([`f69dffd`](https://github.com/eyeonus/Trade-Dangerous/commit/f69dffd66c7c0e2f0b0acb7227bf9d96a9d380fd))

Refactor

* refactor: simplify the system-purge operation to not copy/move data and abuse the db ([`ab75113`](https://github.com/eyeonus/Trade-Dangerous/commit/ab75113ed910ad1301e60376ed0f3d4ae8de6891))