Datalad

🐛 Bug Fix

- BF: git_ignore_check do not overload possible value of stdout/err if present [6937](https://github.com/datalad/datalad/pull/6937) ([yarikoptic](https://github.com/yarikoptic))
- DOCfix: fix docstring GeneratorStdOutErrCapture to say that treats both stdout and stderr identically [6930](https://github.com/datalad/datalad/pull/6930) ([yarikoptic](https://github.com/yarikoptic))
- Explain purpose of create-sibling-ria's --post-update-hook [6958](https://github.com/datalad/datalad/pull/6958) ([mih](https://github.com/mih))
- ENH+BF: get_parent_paths - make / into sep option and consistently use "/" as path separator [6963](https://github.com/datalad/datalad/pull/6963) ([yarikoptic](https://github.com/yarikoptic))
- BF(TEMP): use git-annex from neurodebian -devel to gain fix for bug detected with datalad-crawler [6965](https://github.com/datalad/datalad/pull/6965) ([yarikoptic](https://github.com/yarikoptic))
- BF(TST): make tests use _path_ helper for Windows "friendliness" of the tests [6955](https://github.com/datalad/datalad/pull/6955) ([yarikoptic](https://github.com/yarikoptic))
- BF(TST): prevent auto-upgrade of "remote" test sibling, do not use local path for URL [6957](https://github.com/datalad/datalad/pull/6957) ([yarikoptic](https://github.com/yarikoptic))
- Forbid drop operation from symlink'ed annex (e.g. due to being cloned with --reckless=ephemeral) to prevent data-loss [6959](https://github.com/datalad/datalad/pull/6959) ([mih](https://github.com/mih))
- Acknowledge git-config comment chars [6944](https://github.com/datalad/datalad/pull/6944) ([mih](https://github.com/mih) [yarikoptic](https://github.com/yarikoptic))
- Minor tuneups to please updated codespell [6956](https://github.com/datalad/datalad/pull/6956) ([yarikoptic](https://github.com/yarikoptic))
- TST: Add a testcase for 6950 [6957](https://github.com/datalad/datalad/pull/6957) ([adswa](https://github.com/adswa))
- BF+ENH(TST): fix typo in code of wtf filesystems reports [6920](https://github.com/datalad/datalad/pull/6920) ([yarikoptic](https://github.com/yarikoptic))
- DOC: Datalad -> DataLad [6937](https://github.com/datalad/datalad/pull/6937) ([aqw](https://github.com/aqw))
- BF: fix typo which prevented silently to not show details of filesystems [6930](https://github.com/datalad/datalad/pull/6930) ([yarikoptic](https://github.com/yarikoptic))
- BF(TST): allow for a annex repo version to upgrade if running in adjusted branches [6927](https://github.com/datalad/datalad/pull/6927) ([yarikoptic](https://github.com/yarikoptic))
- RF extensions github action to centralize configuration for extensions etc, use pytest for crawler [6914](https://github.com/datalad/datalad/pull/6914) ([yarikoptic](https://github.com/yarikoptic))
- BF: travis - mark our directory as safe to interact with as root [6919](https://github.com/datalad/datalad/pull/6919) ([yarikoptic](https://github.com/yarikoptic))
- BF: do not pretend we know what repo version git-annex would upgrade to [6902](https://github.com/datalad/datalad/pull/6902) ([yarikoptic](https://github.com/yarikoptic))
- BF(TST): do not expect log message for guessing Path to be possibly a URL on windows [6911](https://github.com/datalad/datalad/pull/6911) ([yarikoptic](https://github.com/yarikoptic))
- ENH(TST): Disable coverage reporting on travis while running pytest [6898](https://github.com/datalad/datalad/pull/6898) ([yarikoptic](https://github.com/yarikoptic))
- RF: just rename internal variable from unclear "op" to "io" [6907](https://github.com/datalad/datalad/pull/6907) ([yarikoptic](https://github.com/yarikoptic))
- DX: Demote loglevel of message on url parameters to DEBUG while guessing RI [6891](https://github.com/datalad/datalad/pull/6891) ([adswa](https://github.com/adswa) [yarikoptic](https://github.com/yarikoptic))
- Fix and expand datalad.runner type annotations [6893](https://github.com/datalad/datalad/pull/6893) ([christian-monch](https://github.com/christian-monch) [yarikoptic](https://github.com/yarikoptic))
- Use pytest to test datalad-metalad in test_extensions-workflow [6892](https://github.com/datalad/datalad/pull/6892) ([christian-monch](https://github.com/christian-monch))
- Let push honor multiple publication dependencies declared via siblings [6869](https://github.com/datalad/datalad/pull/6869) ([mih](https://github.com/mih) [yarikoptic](https://github.com/yarikoptic))
- ENH: upgrade versioneer from versioneer-0.20.dev0 to versioneer-0.23.dev0 [6888](https://github.com/datalad/datalad/pull/6888) ([yarikoptic](https://github.com/yarikoptic))
- ENH: introduce typing checking and GitHub workflow [6885](https://github.com/datalad/datalad/pull/6885) ([yarikoptic](https://github.com/yarikoptic))
- RF,ENH(TST): future proof testing of git annex version upgrade + test annex init on all supported versions [6880](https://github.com/datalad/datalad/pull/6880) ([yarikoptic](https://github.com/yarikoptic))
- ENH(TST): test against supported git annex repo version 10 + make it a full sweep over tests [6881](https://github.com/datalad/datalad/pull/6881) ([yarikoptic](https://github.com/yarikoptic))
- BF: RF f-string uses in logger to %-interpolations [6886](https://github.com/datalad/datalad/pull/6886) ([yarikoptic](https://github.com/yarikoptic))
- Merge branch 'bf-sphinx-5.1.0' into maint [6883](https://github.com/datalad/datalad/pull/6883) ([yarikoptic](https://github.com/yarikoptic))
- BF(DOC): workaround for 10701 of sphinx in 5.1.0 [6883](https://github.com/datalad/datalad/pull/6883) ([yarikoptic](https://github.com/yarikoptic))
- Clarify confusing INFO log message from get() on dataset installation [6871](https://github.com/datalad/datalad/pull/6871) ([mih](https://github.com/mih))
- Protect again failing to load a command interface from an extension [6879](https://github.com/datalad/datalad/pull/6879) ([mih](https://github.com/mih))
- Support unsetting config via `datalad -c :<name>` [6864](https://github.com/datalad/datalad/pull/6864) ([mih](https://github.com/mih))
- Fix DOC string typo in the path within AnnexRepo.annexstatus, and replace with proper sphinx reference [6858](https://github.com/datalad/datalad/pull/6858) ([christian-monch](https://github.com/christian-monch))
- Improved support for saving typechanges [6793](https://github.com/datalad/datalad/pull/6793) ([mih](https://github.com/mih))

⚠️ Pushed to `maint`

- BF: Remove duplicate ds key from result record ([adswa](https://github.com/adswa))
- DOC: fix capitalization of service names ([aqw](https://github.com/aqw))

🧪 Tests

- BF(TST,workaround): just xfail failing archives test on NFS [6912](https://github.com/datalad/datalad/pull/6912) ([yarikoptic](https://github.com/yarikoptic))

Authors: 5

- Adina Wagner ([adswa](https://github.com/adswa))
- Alex Waite ([aqw](https://github.com/aqw))
- Christian Mönch ([christian-monch](https://github.com/christian-monch))
- Michael Hanke ([mih](https://github.com/mih))
- Yaroslav Halchenko ([yarikoptic](https://github.com/yarikoptic))

---

🐛 Bug Fix

- BF(TST): do proceed to proper test for error being caught for recent git-annex on windows with symlinks [6850](https://github.com/datalad/datalad/pull/6850) ([yarikoptic](https://github.com/yarikoptic))
- Addressing problem testing against python 3.10 on Travis (skip more annex versions) [6842](https://github.com/datalad/datalad/pull/6842) ([yarikoptic](https://github.com/yarikoptic))
- XFAIL test_runner_parametrized_protocol on python3.8 when getting duplicate output [6837](https://github.com/datalad/datalad/pull/6837) ([yarikoptic](https://github.com/yarikoptic))
- BF: Make create's check for procedures work with several again [6841](https://github.com/datalad/datalad/pull/6841) ([adswa](https://github.com/adswa))
- Support older pytests [6836](https://github.com/datalad/datalad/pull/6836) ([jwodder](https://github.com/jwodder))

Authors: 3

- Adina Wagner ([adswa](https://github.com/adswa))
- John T. Wodder II ([jwodder](https://github.com/jwodder))
- Yaroslav Halchenko ([yarikoptic](https://github.com/yarikoptic))

---

🐛 Bug Fix

- DOC: minor fix - consistent DataLad (not Datalad) in docs and CHANGELOG [6830](https://github.com/datalad/datalad/pull/6830) ([yarikoptic](https://github.com/yarikoptic))
- DOC: fixup/harmonize Changelog for 0.17.0 a little [6828](https://github.com/datalad/datalad/pull/6828) ([yarikoptic](https://github.com/yarikoptic))
- BF: use --python-match minor option in new datalad-installer release to match outside version of Python [6827](https://github.com/datalad/datalad/pull/6827) ([christian-monch](https://github.com/christian-monch) [yarikoptic](https://github.com/yarikoptic))
- Do not quote paths for ssh >= 9 [6826](https://github.com/datalad/datalad/pull/6826) ([christian-monch](https://github.com/christian-monch) [yarikoptic](https://github.com/yarikoptic))
- Suppress DeprecationWarning to allow for distutils to be used [6819](https://github.com/datalad/datalad/pull/6819) ([yarikoptic](https://github.com/yarikoptic))
- RM(TST): remove testing of datalad.test which was removed from 0.17.0 [6822](https://github.com/datalad/datalad/pull/6822) ([yarikoptic](https://github.com/yarikoptic))
- Avoid import of nose-based tests.utils, make skip_if_no_module() and skip_if_no_network() allowed at module level [6817](https://github.com/datalad/datalad/pull/6817) ([jwodder](https://github.com/jwodder))
- BF(TST): use higher level asyncio.run instead of asyncio.get_event_loop in test_inside_async [6808](https://github.com/datalad/datalad/pull/6808) ([yarikoptic](https://github.com/yarikoptic))

Authors: 3

- Christian Mönch ([christian-monch](https://github.com/christian-monch))
- John T. Wodder II ([jwodder](https://github.com/jwodder))
- Yaroslav Halchenko ([yarikoptic](https://github.com/yarikoptic))

---

💫 Enhancements and new features
- "log" progress bar now reports about starting a specific action as well. [6756](https://github.com/datalad/datalad/pull/6756) (by yarikoptic)
- Documentation and behavior of traceback reporting for log messages via `DATALAD_LOG_TRACEBACK` was improved to yield a more compact report. The documentation for this feature has been clarified. [6746](https://github.com/datalad/datalad/pull/6746) (by mih)
- `datalad unlock` gained a progress bar. [6704](https://github.com/datalad/datalad/pull/6704) (by adswa)
- When `create-sibling-gitlab` is called on non-existing subdatasets or paths it now returns an impossible result instead of no feedback at all. [6701](https://github.com/datalad/datalad/pull/6701) (by adswa)
- `datalad wtf` includes a report on file system types of commonly used paths. [6664](https://github.com/datalad/datalad/pull/6664) (by adswa)
- Use next generation metadata code in search, if it is available. [6518](https://github.com/datalad/datalad/pull/6518) (by christian-monch)

🪓 Deprecations and removals
- Remove unused and untested log helpers `NoProgressLog` and `OnlyProgressLog`. [6747](https://github.com/datalad/datalad/pull/6747) (by mih)
- Remove unused `sorted_files()` helper. [6722](https://github.com/datalad/datalad/pull/6722) (by adswa)
- Discontinued the value `stdout` for use with the config variable `datalad.log.target` as its use would inevitably break special remote implementations. [6675](https://github.com/datalad/datalad/pull/6675) (by bpoldrack)
- `AnnexRepo.add_urls()` is deprecated in favor of `AnnexRepo.add_url_to_file()` or a direct call to `AnnexRepo.call_annex()`. [6667](https://github.com/datalad/datalad/pull/6667) (by mih)
- `datalad test` command and supporting functionality (e.g., `datalad.test`) were removed. [6273](https://github.com/datalad/datalad/pull/6273) (by jwodder)

🐛 Bug Fixes
- `export-archive` does not rely on `normalize_path()` methods anymore and became more robust when called from subdirectories. [6745](https://github.com/datalad/datalad/pull/6745) (by adswa)
- Sanitize keys before checking content availability to ensure that the content availability of files with URL- or custom backend keys is correctly determined and marked. [6663](https://github.com/datalad/datalad/pull/6663) (by adswa)
- Ensure saving a new subdataset to a superdataset yields a valid `.gitmodules` record regardless of whether and how a path constraint is given to the `save()` call. Fixes 6547 [6790](https://github.com/datalad/datalad/pull/6790) (by mih)
- `save` now repairs annex symlinks broken by a `git-mv` operation prior recording a new dataset state. Fixes 4967 [6795](https://github.com/datalad/datalad/pull/6795) (by mih)

📝 Documentation
- API documentation for log helpers, like `log_progress()` is now included in the renderer documentation. [6746](https://github.com/datalad/datalad/pull/6746) (by mih)
- New design document on progress reporting. [6734](https://github.com/datalad/datalad/pull/6734) (by mih)
- Explain downstream consequences of using `--fast` option in `addurls`. [6684](https://github.com/datalad/datalad/pull/6684) (by jdkent)

🏠 Internal
- Inline code of `create-sibling-ria` has been refactored to an internal helper to check for siblings with particular names across dataset hierarchies in `datalad-next`, and is reintroduced into core to modularize the code base further. [6706](https://github.com/datalad/datalad/pull/6706) (by adswa)
- `get_initialized_logger` now lets a given `logtarget` take precedence over `datalad.log.target`. [6675](https://github.com/datalad/datalad/pull/6675) (by bpoldrack)
- Many uses of deprecated call options were replaced with the recommended ones. [6273](https://github.com/datalad/datalad/pull/6273) (by jwodder)
- Get rid of `asyncio` import by defining few noops methods from `asyncio.protocols.SubprocessProtocol` directly in `WitlessProtocol`. [6648](https://github.com/datalad/datalad/pull/6648) (by yarikoptic)
- Consolidate `GitRepo.remove()` and `AnnexRepo.remove()` into a single implementation. [6783](https://github.com/datalad/datalad/pull/6783) (by mih)
🛡 Tests
- Discontinue use of `with_testrepos` decorator other than for the deprecation cycle for `nose`. [6690](https://github.com/datalad/datalad/pull/6690) (by mih bpoldrack) See [#6144](https://github.com/datalad/datalad/issues/6144) for full list of changes.
- Remove usage of deprecated `AnnexRepo.add_urls` in tests. [6683](https://github.com/datalad/datalad/pull/6683) (by bpoldrack)
- Minimalistic (adapters, no assert changes, etc) migration from `nose` to `pytest`.
Support functionality possibly used by extensions and relying on `nose` helpers is left in place to avoid affecting their run time and defer migration of their test setups.. [6273](https://github.com/datalad/datalad/pull/6273) (by jwodder)

Authors: 7

- Yaroslav Halchenko (yarikoptic)
- Michael Hanke (mih)
- Benjamin Poldrack (bpoldrack)
- Adina Wagner (adswa)
- John T. Wodder (jwodder)
- Christian Mönch (christian-monch)
- James Kent (jdkent)

🐛 Bug Fix

- Fix broken annex symlink after git-mv before saving + fix a race condition in ssh copy test [6809](https://github.com/datalad/datalad/pull/6809) ([christian-monch](https://github.com/christian-monch) [mih](https://github.com/mih) [yarikoptic](https://github.com/yarikoptic))
- Do not ignore already known status info on submodules [6790](https://github.com/datalad/datalad/pull/6790) ([mih](https://github.com/mih))
- Fix "common data source" test to use a valid URL (maint-based & extended edition) [6788](https://github.com/datalad/datalad/pull/6788) ([mih](https://github.com/mih) [yarikoptic](https://github.com/yarikoptic))
- Upload coverage from extension tests to Codecov [6781](https://github.com/datalad/datalad/pull/6781) ([jwodder](https://github.com/jwodder))
- Clean up line end handling in GitRepo [6768](https://github.com/datalad/datalad/pull/6768) ([christian-monch](https://github.com/christian-monch))
- Do not skip file-URL tests on windows [6772](https://github.com/datalad/datalad/pull/6772) ([christian-monch](https://github.com/christian-monch))
- Fix test errors caused by updated chardet v5 release [6777](https://github.com/datalad/datalad/pull/6777) ([christian-monch](https://github.com/christian-monch))
- Preserve final trailing slash in ``call_git()`` output [6754](https://github.com/datalad/datalad/pull/6754) ([adswa](https://github.com/adswa) [yarikoptic](https://github.com/yarikoptic) [christian-monch](https://github.com/christian-monch))

⚠️ Pushed to `maint`

- Make sure a subdataset is saved with a complete .gitmodules record ([mih](https://github.com/mih))

Authors: 5

- Adina Wagner ([adswa](https://github.com/adswa))
- Christian Mönch ([christian-monch](https://github.com/christian-monch))
- John T. Wodder II ([jwodder](https://github.com/jwodder))
- Michael Hanke ([mih](https://github.com/mih))
- Yaroslav Halchenko ([yarikoptic](https://github.com/yarikoptic))

---

🐛 Bug Fix

- Prevent duplicated result rendering when searching in default datasets [6765](https://github.com/datalad/datalad/pull/6765) ([christian-monch](https://github.com/christian-monch))
- BF(workaround): skip test_ria_postclonecfg on OSX for now ([yarikoptic](https://github.com/yarikoptic))
- BF(workaround to 6759): if saving credential failed, just log error and continue [6762](https://github.com/datalad/datalad/pull/6762) ([yarikoptic](https://github.com/yarikoptic))
- Prevent reentry of a runner instance [6737](https://github.com/datalad/datalad/pull/6737) ([christian-monch](https://github.com/christian-monch))

Authors: 2

- Christian Mönch ([christian-monch](https://github.com/christian-monch))
- Yaroslav Halchenko ([yarikoptic](https://github.com/yarikoptic))

---