Pg-spot-operator

Latest version: v0.9.12

Safety actively analyzes 688674 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 2 of 3

0.9.5

------------------
- Release: version 0.9.5 🚀 [Kaarel Moppel]
- Merge pull request 97 from pg-spot-ops/add-instance-family-filtering.
[Kaarel Moppel]

Add option to select / filter suitable instances based on instance type family regex
- Reflect the new --instance-family option in the README. [Kaarel
Moppel]
- Implement --instance-family regex based filtering if provided. [Kaarel
Moppel]
- Add --instance-family dummy attribute. [Kaarel Moppel]
- Merge pull request 96 from pg-spot-ops/api-improvements. [Kaarel
Moppel]

API - attribute naming improvements
- API rename: self-terminate -> self-termination. [Kaarel Moppel]
- Admin-user-password -> admin-password. [Kaarel Moppel]
- Admin_user_password -> admin_password. [Kaarel Moppel]
- More postgresql -> postgres. [Kaarel Moppel]
- API: postgresql -> postgres. [Kaarel Moppel]

Nobody really calls it postgresql I guess
- API: floating_ips -> ip_floating. [Kaarel Moppel]
- API: cpu_architecture -> cpu_arch. [Kaarel Moppel]

Just a bit too cumbersome on the CLI
- Disable Github Docker workflow as using / testing Docker Hub
integration. [Kaarel Moppel]
- Merge pull request 95 from pg-spot-ops/cli-add-volume-type-iops-
bandwith. [Kaarel Moppel]

CLI - add flags to set volume type, iops, bandwith
- Wire new volume params to the manifest. [Kaarel Moppel]
- Add new flags: --volume-type, --volume-iops, --volume-throughput.
[Kaarel Moppel]

0.9.0

------------------
- Release: version 0.9.0 🚀 [Kaarel Moppel]
- In CLI input mode infer --region automatically from --zone if not set.
[Kaarel Moppel]

Same as in manifest mode. Otherwise would get still a global price
check in --check-price mode
- Merge pull request 94 from pg-spot-ops/check-price-improvements.
[Kaarel Moppel]

Check price improvements - don't bail when one region's price fetching fails + allow global --check-price
- README update - demo the new global price check. [Kaarel Moppel]

In the Usage via Python section
- Allow global price check with no region set at all. [Kaarel Moppel]
- Improve CLI input validation - don't need --storage-min for EBS
storage. [Kaarel Moppel]

Also can have --instance-name set for --check-price mode
- Fix authenticated / boto3 price resolving. [Kaarel Moppel]
- Show a warning about regions not reached for pricing info. [Kaarel
Moppel]

A la:
WARNING - failed to inquiry regions: ['ap-southeast-5', 'cn-north-1', 'cn-northwest-1']
- Logging - don't show asctime and levelname in --check-price mode.
[Kaarel Moppel]
- Global --check-price improvements - dont bail on one region failing.
[Kaarel Moppel]
- Improve HW reqs change handling. [Kaarel Moppel]

Currently the running instance was terminated but due to caching the
"ensure VM" function didn't pick up and a whole main loop passed before
rebuild was tried
- Retries for --connstr-output-only mode. [Kaarel Moppel]

Currently program exited on first loop errors in --connstr-output-only
mode but no real season for that, cloud is volatile - just keep trying
- Linter Python 3.10 -> 3.12. [Kaarel Moppel]
- Fix new Ansible folder created on each main loop in case of errors.
[Kaarel Moppel]

Can take too much disk space in the end if left running for too long.
Now have oa folder per action per day
- Fix explicit --instance-types input being too aggressive. [Kaarel
Moppel]
- Merge pull request 93 from pg-spot-ops/eviction-rate-strategy.
[Kaarel Moppel]

Add eviction rate based and a balanced instance selection strategy
- In "cheapest" selection mode don't consider the worst eviction rate
bracket still. [Kaarel Moppel]

With >20% eviction rates
- README - add new eviction rate indicator to sample --check-price
output. [Kaarel Moppel]
- Update READMEs + log hints on used strategy. [Kaarel Moppel]
- Make the new "balanced" instance selection strategy the default.
[Kaarel Moppel]
- Add a "balanced" instance selection strategy. [Kaarel Moppel]

Weighed average on price + eviction rate
- Add tests for instance_type_selection.py. [Kaarel Moppel]
- Rename InstanceTypeSelectionDefault to InstanceTypeSelectionCheapest.
[Kaarel Moppel]
- Eviction rate strategy working. [Kaarel Moppel]

Also show the expected eviction rate when we have the information
- WIP refactor selection strategy. [Kaarel Moppel]
- Add AWS public Spot eviction rate parsing. [Kaarel Moppel]

Based on https://spot-bid-advisor.s3.amazonaws.com/spot-advisor-data.json
- README - reduce quickstart --check-price output a bit for readability.
[Kaarel Moppel]

Also hint that --region can be a regex in --help
- README - mention port 22 SG access pre-requisite in Quickstart.
[Kaarel Moppel]

Remove port 5432 mention as not a hard requirement for successful Ansible setup
- README - mention port 22 / 5432 SG access pre-requisite in Quickstart.
[Kaarel Moppel]

0.8.8

------------------
- Release: version 0.8.8 🚀 [Kaarel Moppel]
- README - mention "assume role" based authentication option. [Kaarel
Moppel]

Under pre-requisites to creating a DB
- Cross regional price check (90) [Kaarel Moppel]

* Add util to resolve fuzzy regions to real ones

* Main flow in place - show max top 3 cheapest regions

* Cope with parsing of NA spot prices in S3 price list files

* Update READMEs to reflect the new --region regex + --check-price combo

* Fail early when regex --region input used in non-check-price mode

* Don't allow regex --region also for --teardown / --teardown-region
- New CLI flag: --list-regions (89) [Kaarel Moppel]

* New CLI option: --list-regions

* Document new --list-regions in docs/README_env_options.md
- Show selected instance storage speed class also. [Kaarel Moppel]

Or "EBS only" if no instance storage support
- Uncomment S3 privileges in sample Terraform. [Kaarel Moppel]

As not needed for base functionality

0.8.7

------------------
- Release: version 0.8.7 🚀 [Kaarel Moppel]
- Fix CPU arch "guessing" from instance_type name. [Kaarel Moppel]

Was fixed to ARM. Assuming "g" always present for ARM instances
- Bump codecov/codecov-action from 4 to 5 (87) [dependabot[bot],
dependabot[bot]]

Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
dependency-type: direct:production
update-type: version-update:semver-major
...
- Non auth price check (88) [Kaarel Moppel]

* WIP

* Parse ResolvedInstanceTypeInfo from AWS static ondemand pricing JSON

Some info is not quite as explicit as from boto3 directly, like CPU
arch, so need to do some best effort heuristics until find a better
source for instance details

* WIP

* Generalize instance type info from boto3 and JSON APIs into InstanceTypeInfo

* Linter passing

* Fix ondemand JSON cache reading

* Don't ugly-error when no SKUs match user HW reqs

* Fix CPU arch filtering

Support arm / non-arm only for now

* Fix non-auth price fetching from sa-east-1 region

South America (São Paulo) had to be un-accented

* In debug mode hint if using the S3 API or boto3 calls for pricing

* Respect --instance-types user input

Don't consider any other SKUs then

* Minor formatting / debug output changes

* README - update to new output messages

Plus add an approximate comparison to RDS prices
- Fetch price info via AWS endpoints (67) [Evans Akai Bekoe, Kaarel
Moppel]

* fetch price info via AWS endpoints

* fix linter and errors

* provide else-clause of pricing function

* cast price string to float

* add logging for when pricing info is not found

* Don't double-check the pricing manifest + os.path.join + some logging

* Simplify on-demand price parsing + a test

* WIP

* Pass tests - simplify the on-demand pricing URL derivation

* Use the old ec2.shop for fallback if AWS static info fails

* Round on-demand AWS and fallback price using same precision

* Clean up older than 1 week ondemand pricing files automatically

* Don't fetch the ondemand meta files at all as not likely to change

And we have a fallback in place

---------
- K8s/README_k8s.md typo. [Kaarel Moppel]
- README - add a link to the k8s sub-readme. [Kaarel Moppel]
- Add a K8s readme. [Kaarel Moppel]
- Minor k8s/example_deployment.yaml adjustment. [Kaarel Moppel]
- Helm example - set storageClass to "standard" [Kaarel Moppel]

To make Minikube PVCs work without an explicit PV
- Add some more Docker build exclude folders. [Kaarel Moppel]
- K8s - add a sample Deployment manifest. [Kaarel Moppel]
- K8s - add a minimal Helm chart (85) [Kaarel Moppel]

* Add a minimal Helm chart

* Make basic skel work

* Add PVC for .ssh

* Add a PV to make minikube happy

* Silence "image has non-numeric user (nobody)" errors

100Mi -> 10Mi SSH volume

* Add an initContainer to deployment

To fix /app/.ssh permissions that get set to root on volume mount. There
must be a better way though??
- Docker - generate SSH keys properly during runtime (84) [Kaarel
Moppel]

* Docker - generate SSH keys during runtime

Add a dump-init entrypoint wrapper for that

* Set Docker user ID to 5432 to appease PodSecurityPolicy runAsNonRoot

Also remove some root useful extra packages

* Create /app/.ssh during image build
- README - remove Docker examples with bind mounts. [Kaarel Moppel]

As most probably will now run into user privilege issues after switching
to a non-root image + safer conceptually also to feed in only what is
required
- Docker - fix non-root user Ansible SSH access. [Kaarel Moppel]

Seems HOME for the nobody user resolved to /nonexistent. Thus
create a soft-link to /app

debug1: Trying private key: /nonexistent/.ssh/id_rsa
- Ansible - do not cache SSH signatures on connect. [Kaarel Moppel]

Better for long-term setups
- Increase SSH ConnectTimeout to 3s from 1s for the displayed SSH
connstr. [Kaarel Moppel]
- Make --connstr-output-only --vm-only aware. [Kaarel Moppel]

Print out the SSH "connstr" instead of Postgres connstr
- Docker - switch to a non-root image (83) [Kaarel Moppel]

As things seems to have matured enough
- README - replace Github release badge with PyPI release. [Kaarel
Moppel]
- README - add a few badges. [Kaarel Moppel]
- README - fix double credentials setting in the Docker example. [Kaarel
Moppel]
- Docker image building - point to Containerfile explicitly. [Kaarel
Moppel]
- Automate Docker image building / publishing. [Kaarel Moppel]
- Region teardown improvement - mark all instances as deleted also in
CMDB. [Kaarel Moppel]

Consider any CMDB errors as non-critical though
- Fix --teardown-region - not all instances were cleaned up. [Kaarel
Moppel]

The describe_instances loop didn't account for multiple reservations
- VM provisioning - increase OS disk from default 8 to 20 GB. [Kaarel
Moppel]

Previous change wasn't effective as root device for Debian AMI is
actually /dev/xvda not /dev/sda1
- Docs/README_development.md - link to the Resource Explorer img.
[Kaarel Moppel]
- README_development.md - mention AWS Resource Explorer. [Kaarel Moppel]

To track down any operator created objects if needed
- README_integration.md - note that callbacks are limited to 30 seconds
runtime. [Kaarel Moppel]
- A new top level manifest attribute "vm_only" to skip the Postgres
setup (82) [Kaarel Moppel]

* New top level manifest attribute: vm_only

To skip the Postgres setup

* Update the manifest in CMDB after vm_only loop also

As in normal mode, not to get superfluous manifest diff output
- Hint that non-floating IPs take longer to recover after an eviction.
[Kaarel Moppel]

Due to some AWS NIC state refresh lag, can't re-attach before NIC shown
as available again
- Teardown fixes (81) [Kaarel Moppel]

* Delete only explicitly created NICs on teardown

Also only EIPs of the target instance

* Delete NICs before EIPs also in teardown_region()

As EIPs depend on NICs

* Don't even try to delete backup if we don't have a bucket set
- Mention our VPC + IAM setup Terraform scripts also in the AWS CLI
readme. [Kaarel Moppel]
- README - suggest to run the Terraform from "scripts" if no play
account. [Kaarel Moppel]

available for the user
- Fix last one - add missing EIP policies. [Kaarel Moppel]
- Example Terraform to create a sandbox VPC + IAM user + creds (80)
[Kaarel Moppel]

IAM policies are region limited for additional play safety
- READEME - switch quickstart demo from Docker to Py as more terse.
[Kaarel Moppel]
- New CLI flag: --aws-key-pair-name / PGSO_AWS_KEY_PAIR_NAME (78)
[Kaarel Moppel]

* New CLI flag: --aws-key-pair-name / PGSO_AWS_KEY_PAIR_NAME

To specify an existing EC2 SSH key pair to access the VM. Other SSH
inputs also still effective

* Don't bail when aws.key_pair_name input is invalid

As other SSH key specification options could be valid

* README - in Docker howto showcase the new PGSO_AWS_KEY_PAIR_NAME envvar
- README - make more clear which AWS credentials are required. [Kaarel
Moppel]

In the quickstart section

0.8.6

------------------
- Release: version 0.8.6 🚀 [Kaarel Moppel]
- Don't display the main loop sleep message in dry-run mode. [Kaarel
Moppel]
- Auto-download ansible setup scripts from GitHub if missing (76)
[Kaarel Moppel]

* Auto-download the Ansible setup scripts from Github

If not found locally and --ansible-path not set. To make pipx usage
more user-friendly

* More Github downloading to util.py
- Security readme - recommend a new isolated VPC for running the
operator. [Kaarel Moppel]
- Add a separate Readme for CLI options (70) [Kaarel Moppel]

* Add a new help doc on CLI / ENV input parameters

* Move to "definition list" format for readability

* Revert "Move to "definition list" format for readability"

This reverts commit 93397a0935e6d772b77d61ac209bae5dbf9d6850.

As seems Github does not support Markdown definition lists yet :(

* Fix a line
- README - mention the support for Ansible Vault secrets. [Kaarel
Moppel]
- README - add a section on extensions usage. [Kaarel Moppel]
- README typos + minor wording. [Kaarel Moppel]

Move the project status section to footer
- README update - more compact quickstart. [Kaarel Moppel]
- Mask long and ugly AWS API timeout errors in non-verbose mode. [Kaarel
Moppel]

When listing active VMs. As quite common actually on flaky network.
- README usage section - link to all options. [Kaarel Moppel]

And add an example parameter to enable the pgvector extension
- Refactor cleanup helper (68) [Kaarel Moppel]

* Clean up resources region by region

More logical so

* Loop regions by sorted region name

* Dont run the script without any parameters

Show usage instead if no params given
- Remove some noise from the README. [Kaarel Moppel]

Some sections actually linked in footer

0.8.5

------------------
- Release: version 0.8.5 🚀 [Kaarel Moppel]
- README - remove instance name from the quickstart price check example.
[Kaarel Moppel]

As not required since prev commit
- Don't require --instance-name input in --check-price mode. [Kaarel
Moppel]
- Don't store user provided secrets in plain-text in the CMDB (64)
[Kaarel Moppel]

* Don't store any user secrets in the CMDB

As per user feedack. Carries some risk still although mostly a dev tool

* Refactor the connect string printing to a more logical place

As per PR review https://github.com/pg-spot-ops/pg-spot-operator/pull/64

* Rebase
- In --check-price mode show the spot price in any case. [Kaarel Moppel]

Even if can't get the On-Demand price for a discount comparison
- Docs reorg - split the README (66) [Kaarel Moppel]

* Add a short primer on AWS CLI basics

* Split main README into smaller ones by topic

* Check linking

* Link all sections

* Add a note on project status

* Make 1st lines of README more light to read

Plus don't downplay the security posture, as still accounted for

* AWS basics - add a link to account creation

* Change docs base path from reorg branch to main
- Delete tmp Ansible folder after success (65) [Kaarel Moppel]

To lower the likelyhood of leaking any passwords
- Allow using non-default VPCs (60) [Kaarel Moppel]

* New manifest attribute / CLI flag --aws-vpc-id

To be able to use non-default VPCs more conveniently. Previously once
could already do it by specifying a Subnet ID - but this assumes also
the AZ is set correctly by the user. Now we select the subnet according
to the cheapest AZ found if just VPC ID specified

* README - add information on the new --aws-vpc-id flag

Plus move the whole Security section higher, as obviously a topic
nowadays

* README - recommend a new VPC for all operator instances

* README - add example information how to create a new play VPC

Toghether with SG rules opening up ports used by the operator

* Rename --public-ip to --assign-public-ip

As per PR feedback

* Specify what a public instance means
- Increase after-VM-create sleep from 5s to 10s. [Kaarel Moppel]

Seems reducing after-VM-create sleep from 30s to 5s was too optimistic
still
- Update README.md. [Evans Akai Bekoe]

Page 2 of 3

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.