CLI
- fix file name and message when extracting a file from an archive !4602
- templates: circumvent index wrap around for aliases !4603
- multi-object transform and copy: continue-on-err; add similar keywords !4605
- `show object` (and its variations): improve usability and fix minors !4674
- `show config` improvements; follow-up !4676
- add command to immediately unregister node from cluster map !4693
- new command `ais storage cleanup` !4704
- `attach mountpath` and `detach mountpath` (ref) !4705
- `show storage` subcommands !4713
- `--wait` option for storage cleanup command !4773
- report human-readable stats in `show job xaction` !4788
- add commands to manage AIS CLI configuration !4791
- improve `alias` command and docs !4795
- follow-up: introduce `xaction snapshot` (aka "snap") into CLI templates !4806
- add `ais cluster show stats` subcommand !4807
- refresh for stats !4811
- improve `job xaction show` command; minor refactoring !4815, !4816
- node IDs are very special completions (provide visual cue) !4823
- fix archive ls; set and show config usability; log_level !4824
- remove glog's `vmodule` (obsolete) !4826
- improve `ais cluster rebalance` command, allow `ais archive ls` to list entire bucket (of tarballs, etc.) !4828
- do not print hint if JSON format is on !4839
- introduce `unknown-value` and `not-set-value`, and use consistently; fix `stop-maintenance` logic !4842
- downloader: fix error handling, improve usability, !4844
Integration and unit testing; CI/CD
- add Go 1.17 shuffling for short tests !4615
- restore original primary in `TestForwardCP` !4619
- use new `t.Setenv` instead of `os.Setenv` and `defer os.Unsetenv` !4620
- CI: fix lint errors on MacOS !4622
- wait for `xaction` to finish when enabling mirroring !4628
- tests/cleanup: ignore "server closed idle connection" !4631
- fix failing `TestGetAfterReregisterWithMissedBucketUpdate` !4640
- dev-tools/`tassert`: timestamp failure message, customize print-stack !4647
- dev-tools and tests: continued refactoring and cleanup !4654, !4656
- CI: add minimal cluster job that runs with 1 proxy and 1 target !4643
- save/check cluster state; log: memory reporting !4677
- CI: skipping tests in short mode !4776
- test cluster config persistence and `mountpath` add/remove across restarts !4821
- massive read operation vs detach/attach `mountpath` (non-redundent bucket) !4822
- wait for mirroring, re-enable `mountpaths` !4835
Documentation & blog; website [aiatscale.org](https://aiatscale.org/)
- transition to Go 1.17 (add a note) !4623
- remove empty reference to Python client !4636
- remote AIS cluster: examples and references !4665
- remote buckets (naming notation, CLI examples, cross-refs) !4669
- bucket property inheritance, LRU note; E2E tests: `set-custom` !4673
- configuration.md; transport/Rx: PDU header validation !4679
- website: AIStore & ETL: Using AIS/PyTorch connector to transform ImageNet (post 2) !4694
- remote ais cluster, global namespace !4697
- cleanup `http_api` docs and add missing json for `list buckets` operation !4715
- website: AIStore & ETL: Introduction (post 1) !4716
- update command-line usage !4727
- `ais cluster show stats` !4809
- update `ais show job` CLI !4817
- website: copying existing file datasets in two easy steps !4829
- update main v3.8 README !4830
- touch cluster.md; bump version to 3.8 !4838
- [What's new in AIS v3.8](https://aiatscale.org/blog/2021/12/15/whats-new-in-v3.8)
Build and toolchain; dependencies
- transition to Go 1.17 and use new Go features !4627
- make: add `run` and `restart` make targets !4707
- reinitialize go-modules and upgrade all minor versions !4724, !4726
Bug fixes and improvements
- multi-object copy/transform: handle errors; ref common (part two) !4601
- multi-object archive/copy/transform: aborting logic and cleanup !4606
- general: use Go 1.17 enhancements !4607
- health-checker: more efficient method to read directory entries !4608
- bucket copy/transform: aborting logic, cleanup, and test !4609
- transport/streams: fix broadcast in a single-target cluster !4610
- revise `err-aborted`, refactor common errors, downloader !4611
- cluster config vs bucket props vs LRU !4612
- erasure-coding: fix bucket-encoding xaction hang on errors and upon abort !4613
- deployment: add registry URL in `docker build` and `docker push` commands !4614
- add atomic counted err value !4616, !4617
- on-demand `xaction`: mutex to protect compound state !4618
- `etl`: add more details when K8s pod times out on being ready; add debug log !4621, !4624
- intra-cluster notifications: minor refactoring !4625
- `xaction`: add immutable (original) bucket !4626
- revise xaction aborting logic !4629
- `lru` pkg: trash non-existing buckets; health check: docs and follow-up ref-s !4630
- storage cleanup: remove artifacts of erasure-coding (unfinished slices, redundant replicas and metafiles) !4632
- `etl`: fix waiting for a ready condition when starting K8s pod !4633
- fix global-rebalance `can-start` & `must-run` helpers !4634, !4635
- log: add `backend` module; AWS: ignore "unknown region" most of the time !4637
- single-target cluster: support bucket renaming !4638
- node name vs 1) nodes joining the cluster and 2) early startup !4639
- `xaction` pkg: consistent naming and refactoring !4641
- refine health check: add 'ready to rebalance'; fix test to wait for cluster state !4642
- default bucket props (ref) !4644
- glog: set-node, reduce header; transport streams: add destination ID !4645
- copy configuration values for default bucket props !4646
- retriable connection errors; transport: do retry !4648
- `etl`: remove single-pod limitation (i.e., the capability to run multiple ETLs) !4649
- retriable connection errors: use the same condition consistently throughout !4650
- single-target cluster shutdown and assorted fixes !4651
- global rebalance: refine preemption logic, abort associated streams (major ref) !4652, !4653
- custom object metadata: set/replace vs add/update !4655
- consistent custom versioning, checksumming, ETag (major refactoring) !4657
- default buffer sizes (TCP, HTTP) !4658
- custom metadata: unify downloader; unify version comparison !4659, !4661
- downloader: make object comparison much more rigorous !4662
- object metadata: to/from header converters !4663
- unify the code that checks local/remote equality, change `backend.HeadObj()` API, simplify cold-GET !4664
- erasure-coding config and bucket props: add batch size back for compatibility !4666
- object metadata: system attributes (as opposed to user-defined) !4667
- object metadata: unset custom keys on PUT, resolve bucket copy vs `inc-version` dichotomy !4668
- unify object props and system object attributes (major revision) !4670
- get-bucket-props: remove redundant code, simplify !4671, !4672
- configuration: add `transport-idle-teardown` !4675
- `memsys`: tracking memory stats and responding to OOM (major rev) !4678
- `xmeta` (tool): add `VMD`, refactor !4680
- `etl`: add new (non-HTTP) communicator and stop relying on the K8s API Server !4681
- `SizeBytes()` and `AtimeUnix()` methods for CT !4682
- `volume`: revise, refactor, and reinforce operations on metadata !4683, !4684
- `downloader`: refactoring !4685
- cluster membership: shutdown, decommission, maintenance !4686
- target standby mode !4688, !4689, !4691
- dev scripts: deploy remote ais cluster with the same build tags !4690
- refactor and simplify admin-join/auto-join logic !4692
- follow-up: self-join, admin-join, immediate removal from the cluster map !4695
- phase out "unregister" & further consistent renames !4696
- add volume package (ref) !4698
- `volume`: consolidate loading and initialization; bootstrap in two passes !4699
- `cmn/context` => `cmn/cos/context`; rename "attach/detach remote ais" (minor ref) !4700
- `volume`: keep it sync with local config !4701
- `etl`: add more Python runtimes !4702
- `volume`: keep it sync with local configuration !4703
- api to attach/detach/enable/disable `mountpaths`: pass node ref consistently !4706
- make: add `run` and `restart` targets !4707
- error formatting (ref) !4708
- api package (major ref) !4709, !4710
- "easy URL": accommodate list-objects with no json msg !4711
- deploy script: fix getting the last parameter !4712
- intra-cluster reverse proxying vs content-length !4714
- improve error formatting and content !4717, !4718
- `cluster/hrw` and `cmn/err`: continued refactoring !4719
- graceful `mountpath` (disk) removal - major revision !4721, !4722
- `fs`: more validation and naming consistency !4725
- `api` to attach `mountpath` with `force` !4728
- transactional (begin -- commit) to remove `mountpath`, with committing after resilvering !4729
- lint and refactoring, housekeeping !4730
- follow-up: add wait-for-resilver, sort `mountpaths`; increase CI timeout !4732
- add/remove disks: make it transactional with resilvering in-between !4735, !4736, !4740, !4744
- refactor and split `reb` pkg, add `res` (resilver) pkg !4737
- revise `GFN` (state): simplify global, move/rewrite local !4738
- follow-up: move global GFN state to reb pkg !4739
- separate storage cleanup xaction from LRU one !4741
- `xaction` registry vs housekeeper: tweak initializations !4742
- `housekeeper`: run first & reinforce; x-factories: ordered initialization !4743
- `cos` pkg: make `ExitLogf` print file and line of the caller !4745
- `housekeeper`: add wait for housekeeper readiness when registering !4746
- `fs`: `get` and `get-available`; `test-init`; refactoring !4747
- storage cleanup (feature, major upgrade) !4748
- rename API action constants for consistency; tweak OOS logic !4749
- `dsort`: fix panic when creating shard !4750
- new pkg to combine both cleanup and eviction logic !4751
- `memsys`: reduce free-mem requirement for tests & minor fixes !4752
- storage cleanup: repackage, rewrite, and refactor !4753, !4754
- `memsys` for tests (follow-up) !4755
- parse request query only once (core) !4757
- create and lookup buckets on the fly (major rev) !4758
- `memsys` and `housekeeper`: further clarification !4759
- follow-up: do not lookup remote bucket; stopping hk !4760
- revise and refactor logic to handle `deleted` (formerly `trash`) !4761
- `memsys` vs mock target: more orderly initialization !4762
- move mock target out of production code !4763
- `memsys`: tune-up low-watermark !4764
- delete/undelete: retain bucket names when deleting !4766
- `memsys`: two distinct pairs of slab allocators !4767
- general: rename `mountpoint` to `mountpath` for consistency !4768
- `memsys`: tune-up for OOM !4769
- `memsys`: reintroduce max-depth !4770
- reduce glog and tweak `panic-OOM` condition !4771
- `memsys`: fix swapping state !4772, !4774
- `fs`: make `mountpath` flags atomic !4775
- `xreg` pkg: remove obsolete `RebalanceArgs` struct !4777
- `ais` pkg: split `etl.go` into `tgtetl.go` and `prxetl.go` for consistency !4778
- `cmn` pkg: sort actions alphabetically !4779
- `etl`: integrate etl communicator with `xaction` !4780
- `set-name` when joining nodes; keepalive logging; test tweaks !4781
- not aborting `xactions` anymore when adding/removing `mountpaths` !4782
- `fs`: refactor fqn parsing !4783
- `fs`: refactor fqn parsing, update unit tests and docs !4784, !4785
- log rotation: uniformity of the headers !4786
- `fs`: minor fixes; tests !4787
- add an option to remove `mountpaths` without resilvering !4789
- `etl`: manage etl metadata !4790
- refactor `xaction` stats: remove redundant interfaces, rename !4792
- `xactions`: introduce snapshot, add in and out counters !4793, !4794
- integrate: streams and target stats, rebalance and `xaction` snapshots !4796
- `dsort`: wait for all in-flight stream requests !4797
- data mover to support xaction stats; in, out, and locally processed !4798
- `transport`: complete transition to using opcodes !4799
- `stats` pkg: add `get-daemon-stats` API, remove redundant code and refactor !4800
- `etl`: fix panic in unit test !4801
- `etl`: change signature of `InitSpec` function to require options !4802
- `memsys`: tweak init-once logic yet again !4803
- consolidate all mock objects in cluster/mock !4804
- `xactions`: complete transition to snaps (ref) !4805
- `dsort`: fix placement of aborted check !4808
- fix streams received-bytes count; bump CLI and aisloader versions !4810
- remove logic to compute min/max k-alive latency; simplify & refactor !4812
- capability to recover lost or corrupted node ID at startup !4813, !4814
- detach/disable mountpath: actually wait for resilvering !4818
- uuid/gid: consolidate and refactor generation and validation !4819, !4820
- aistore with no disks (feature) !4825
- http provider: ETag check and prefetch prefix !4827
- etl metadata: minor fixes and ref !4831
- rebalance: simplify-out Tx semaphore !4832
- always re-encode etl metadata; improvements and ref !4833
- `volume`: when creating new volume (and associated VMD), make sure to have a record of configured `fspaths` !4834
- `cmn` pkg: add is-err-not-found !4836
- option to force starting up with a lost or missing mountpath !4837
- change one api constant to support rolling upgrade !4840
- fix `ais advanced remove-from-smap` command !4841
- `minikube`: RBAC v1 !4843