Xonsh

====================

**Fixed:**

* fixed regression issue in loading `xontrib-abbrevs <https://github.com/xonsh/xonsh/pull/4757>`_
* Allow xonsh to start gracefully even if modal cursors aren't in the available
prompt_toolkit version

**Authors:**

* Gil Forsyth
* Noortheen Raja

====================

**Added:**

* Added interface to complete any alias that has ``xonsh_complete`` attribute. It is a function with ``fn(**kwargs) -> Iterator[RichCompletion | str]`` signature.
* added ``$ALIAS_COMPLETIONS_OPTIONS_LONGEST`` to control showing options in completions
* added ``$CMD_COMPLETIONS_SHOW_DESC`` environment variable to control showing command completions with a description part.
* `completer complete` command is added to test current completions
* completions from man page will now show the description for the options if available.
* ``$XONSH_COMPLETER_DIRS`` to put command completers
* ``Aliases.register`` to register an alias function.
* Tracebacks are now printed in color if available (interactive session with shell that supports colors with pygments installed and $COLOR_RESULTS enabled)
* Added python's match statement for python >=3.10.
* Added support for the $SHLVL environment variable, typed as int, using bash's semantics.
* Python files with command completions can be put inside ``xompletions`` namespace package,
they will get loaded lazily.
* `xontrib.fish_completer` is available to complete using `fish` shell.
* Support for pythons sys.last_type, sys.last_value, sys.last_traceback.
* added ``xonsh-uname`` command to ``xoreutils``
* auto-completion support for commands : ``source-foreign``, ``source-bash``, ``source-zsh``, ``source-cmd``
* added ``history transfer`` command to transfer history entries between backends.
* now ``$PROMPT_FIELDS`` is a custom class with method ``pick(field_name)`` to get the field value efficiently.
The results are cached within the same prompt call.
* new class ``xonsh.prompt.base.PromptField`` to ease creating/extending prompt-fields
* **Sublime Text 4** extension to the Editors page.
* Support for the `virtualenv <https://virtualenv.pypa.io/en/20.0.1/extend.html#activation-scripts>`_ ``activate.xsh`` script is back! Ensure you create the virtualenv from the same python where xonsh is installed.
* vox new/create accepts a new ``--prompt`` argument, which is passed through to ``python -m venv``
* New set of commands and options to manage virtualenvs inspired from ``pew``

* runin
* runinall
* new

* ``--link`` : to associate venv with project directory
* ``--temp`` : to create temporary virtualenvs

* activate

* now will cd into project directory if the venv is associated

* toggle-ssp - toggle system site packages
* project - manage project path associations
* wipe - to quickly remove all user installed packages
* ``prompt.env.env_name`` is now aware of the "prompt" key in ``pyvenv.cfg`` - search order from first to last is: ``$VIRTUAL_ENV_PROMPT``, ``pyvenv.cfg``, ``$VIRTUAL_ENV``, $``CONDA_DEFAULT_ENV``
* new command ``vox upgrade``
* ``xonfig web`` can now update ``abbrevs/aliases/env-variables``.
* Added `xontrib-default-command <https://github.com/oh-my-xonsh/xontrib-default-command>` to xontrib list.
* new `xontrib-django <https://github.com/jnoortheen/xontrib-django>`_ for django management completions
* Added `xontrib-gruvbox <https://github.com/rpdelaney/xontrib-gruvbox>` to xontrib list.
* Added `xontrib-up <https://github.com/oh-my-xonsh/xontrib-up>` to xontrib list.

**Changed:**

* BREAKING CHANGE: ``/etc/xonshrc`` location for run control file has been deprecated in favor of ``/etc/xonsh/xonshrc``.
* Both ``*.xsh`` and ``*.py`` files inside ``$XONSHRC_DIR`` will get loaded now.
* Environment-variables of no predefined type or path environment variables are now represented as strings via the empty string.
* Made stacktraces behave like in python, i.e. when something in user-provided code fails (both interactively and non-interactively), only that part is shown, and the (static) part of the stacktrace showing the location where the user code was called in xonsh remains hidden. When an unexpected exception occurs inside xonsh, everything is shown like before.
* run_compiled_code, run_script_with_cache, run_code_with_cache now return sys.exc_info() triples instead of throwing errors
* SyntaxError tracebacks now by default hide the internal parser state (like in python); set XONSH_DEBUG >= 1 to enable it again.
* XonshError tracebacks now by default hide xonshs internal state; set XONSH_DEBUG >= 1 to enable it again.
* run_code_with_cache takes a new parameter display_filename to override the filename shown in exceptions (this is independent of caching)
* Update uptime lib by the last one from Pypi
* ``umask``, ``ulimit`` commands will not override the system's commands unless requested
* Xontribs that require other third party packages are moved to its own packages.
The following xontribs are moved and can be loaded after install as usual

* mpl
* distributed
* jupyter-kernel
* jedi
* Xonsh adopts `NEP-0029 <https://numpy.org/neps/nep-0029-deprecation_policy.html>`_ in supporting Python versions.
* Privatise certain attributes of lexer/parser to minimise API surface
* Make `XSH.load` calls explicit (not in Execer)
* Make import hooks require Execer
* Simplified foreign functions
* Updated tutorial.rst to clarify use of time_format
* ``vox new`` will use default python version of the system rather than the one vox is run with
* ``xonfig web`` now shows latest xontribs available from ``xonsh.xontribs_meta``

**Removed:**

* ``$XONSH_GITSTATUS_*`` is removed
since the prompt fields can be customized easily now individually.
* ``$XONSH_GITSTATUS_FIELDS_HIDDEN`` is removed.
Please set hidden fields in ``$PROMPT_FIELDS['gitstatus'].hidden = (...)``
* Removed ``xonsh.ptk2`` module whcih was kept for some old packages sake. Now xonsh requires atleast ptk3 version.

**Fixed:**

* Some of the bash completions scripts can change path starting with '~/' to `/home/user/` during autocompletion.
xonsh `bash_completions` does not expect that, so it breaks autocompletion by producing paths like `~/f/home/user/foo`.
After the fix if bash returns changed paths then `/home/user` prefix will be replaced with `~/`.
* ``pip`` completer now handles path completions correctly
* SyntaxErrors thrown during compilation (i.e. not during parsing) now include the offending source line.
* If a .xsh file is imported, the resulting module will now always have an absolute \_\_file\_\_ attribute to be consistent with pythons behavior since python 3.4.
* ``$CONDA_DEFAULT_ENV`` is now respected when xonsh is run outside of conda.
* Fixed unpacking of dictionaries inside a dictionary
* Empty or comments only .xsh files can now be imported to align with pythons behavior.
* Fixed regex globbing for file paths that contain special regex characters (e.g. "test*1/model")
* Fixed list comprehension in return statement incorrectly being parsed as a subprocess command.
* Fixed the expansion of $XONSH_TRACEBACK_LOGFILE user paths (e.g. "~/log")
* Fixed DeprecationWarning when providing autocompletion for a non-callable type with ``(``
* OSC codes in ``$PROMPT`` is no longer removed when using ptk shell.
These codes need to be escaped with ``\001..\002`` instead.
* Attempt to show a modal cursor in vi_mode (ie. block in cmd, bar in ins)
* Xonsh can now be used in VIM (e.g. by ":read !ls" if VIM is configured to use xonsh. This may be the case when xonsh is the default shell.)
* Fixed OSError on Windows when GnuWin32 is installed in the PATH.
* Do not show welcome message when any ``$XONSHRC_DIR`` directory entry exists.
* SyntaxErrors now get initialized with all available fields so that the error message can be formatted properly.
* Raising BaseException no longer causes Xonsh to crash (fix 4567)
* Exceptions in user code when using xonsh non-interactively no longer simply crash xonsh, rather a proper stacktrace is printed and also postmain() is called.
* Tracebacks will now show the correct filename (i.e. as in python) for interactive use "<stdin>", scripts read by stdin "<stdin>" and -c commands "<string>". (Instead of MD5 hashes as filenames or "<xonsh-code>")
* Default ZSH FUNCSCMD was not working in ZSH 5.8 (and possibly other versions)
* Passing multiple files to be sourced to source-foreign was broken
* prompt field ``current_branch`` will now work empty git repository.

**Authors:**

* Gil Forsyth
* Noortheen Raja
* anki-code
* Daniel Shimon
* Peter Ye
* Jason R. Coombs
* dev2718
* Evgeny
* Angus Hollands
* omjadas
* Oliver Bestwalter
* Samuel Dion-Girardeau
* Ryan Delaney
* E Pluribus Unum
* ylmrx
* Hierosme
* Kyllingene
* zzj
* Daniel
* Ganer
* mattmc3
* Evan Hubinger

====================

**Added:**

* added new utility classes ``xonsh.cli_utils.ArgParserAlias``, ``xonsh.cli_utils.ArgCompleter``.
These are helper classes, that add coloring and auto-completion support to the alias-commands.
* when ``$ENABLE_ASYNC_PROMPT=True`` lazy load ``prompt-toolkit``'s color-input support.
* Add ``CTRL-Right`` key binding to complete a single auto-suggestion word.
* Show environment variables' type and descriptions when completing them.
* Add ``CTRL-Backspace`` key binding to delete a single word via ``$XONSH_CTRL_BKSPC_DELETION``.
* Improved ``pip``/``xpip`` completer.
* Separator used by gitstatus can now be styled using ``XONSH_GITSTATUS_SEPARATOR``.
* Complete 'import' statements with modules that aren't loaded.
* Complete multiple modules/objects in 'import' statements.
* Multiple new metadata fields in ``setup.py``
* Pure Python control files are now supported when named ``*.py``.
Using python files may lower the startup time by a bit.
* new environment variable ``$XONSH_TRACE_SUBPROC_FUNC``
to handle ``$XONSH_TRACE_SUBPROC`` output
* Added `xontrib-pyrtn <https://github.com/dyuri/xontrib-pyrtn>` to xontrib list.

**Changed:**

* Display error message when running `xonfig colors` in a non-interactive shell
* Using ``ArgparserAlias`` for ``dirs``, ``popd``, ``pushd``
* use ``ArgparserAlias`` for ``disown`` alias with completion support
* ``history`` alias now has colored help message and completion support when running interactively.
* using ``ArgparserAlias`` for ``trace`` alias with completion support
* improve ``vox`` CLI completions
* use ArgparserAlias for ``xexec``. Now it supports completions.
* ``xonfig`` now has colored help message when ran interactively.
* Using ``ArgparserAlias`` to improve ``xontrib`` completions
* Changed !() to also capture background subprocesses
* Suggested commands are cached for better performance.
* Improved pipelines performance by using a mutable buffer.
* Curly braces { } in directory names are now escaped in the prompt
* The ``--rc`` argument is extended to support directories as well as files.
Passing a directory will result in all ``*.xsh`` files in the directory being
sorted and loaded at startup (equivalent to using the environment variable
``XONSHRC_DIR``).
* The environment variables ``XONSHRC`` and ``XONSHRC_DIR`` are no longer updated by xonsh on
startup according to which files were actually loaded. This caused problems if xonsh is called
recursively, as the child shells would inherit the modified startup environment of the parent.
These variables will now be left untouched, and the actual RC files loaded (according to those
variables and command line arguments) can be seen in the output of ``xonfig``.
* Replaced `xontrib-linuxbrew <https://github.com/eugenesvk/xontrib-linuxbrew>`_ with `xontrib-homebrew <https://github.com/eugenesvk/xontrib-homebrew>`_, which also supports Homebrew on macOS

**Removed:**

* Completely dropped the deprecated ``--config-path`` argument, which no longer
did anything.
* The environment variable ``LOADED_RC_FILES`` is no longer set. It contained a list of booleans
as to which RC files had been successfully loaded, but it required knowledge of the RC loading
internals to interpret which status corresponded to which file. As above, the (successfully)
loaded RC files are now shown in ``xonfig``.

**Fixed:**

* Add quotes in autocomplete when filename contains brackets
* Handle ``None`` value on XSH.env if ``$UPDATE_OS_ENVIRON`` is set to ``True``
* Implemented `__hash__` method to Env, so that it can be used in `lru_cache` without crashing.
* Make sure aliases are always captured regardless of ``$XONSH_CAPTURE_ALWAYS``
* ``fromdircolors`` doesn't crash if output from subprocess call to ``dircolors`` returns
nothing (usually due to permission errors)
* Fixed issue with environment not being iterable on session objects.
* Fixed issue where environment is None in commands cache.
* ``${...}.swap()`` can be called from multiple threads safetly.
* Piping multiple function aliases doesn't raise a recursion error anymore.
* Fixed detection of App Execution Alias for latest 3.8 and 3.9 releases
* ``Jedi`` completer doesn't complete paths with ``~``.
* Sometimes the completion menu doesn't take space when cursor is at the bottom of the screen.
* vox now passes system-site-packages option
* Fix Duplicate paths left over when add paths to Path via xonsh.tools.EnvPath
* Fix Crash with FileNotFoundError when current working directory is deleted 4467
* Completing a single-arg python code segment (e.g. ``(/etc/hos<TAB>)``).
* Fixed pipelines in WSL2
* Newline symbols in Prompt-toolkit's completions are replaced by <space>
* Fix launching processes on Windows by using full paths (https://bugs.python.org/issue8557)

====================

**Fixed:**

* ``execx`` and ``xonsh -c`` previously exposed xonsh-internal code in global scope. They also did not support defining variables and then referring to them in comprehensions, generators, functions, or lambdas. - https://github.com/xonsh/xonsh/issues/4363
* Short color token names can be used in ``register_custom_style()`` (4339)

**Authors:**

* Gyuri Horak
* Jeremy Schlatter

====================

**Added:**

* Added ability to set XONSH_HISTORY_FILE before loading the history backend.
* Added ability to get the arguments list in ExecAlias using ``$args`` and ``$arg<n>`` environment variables.
* Added instruction how to run xonsh AppImage on Alpine
* Xonsh now supports generators as completer functions.
* Completion Context - Allow completers to access a parsed representation of the current commandline context.
* Added casting CommandPipeline to int, hash and str.
* Ability to call the tool by the name from callable alias with the same name without the infinite loop error.
* ``on wsl`` field when running xonfig (when linux is detected)
* Help and superhelp (``obj?`` and ``obj??``) now use the ``__name__`` if available.
* added ``$XONSH_GITSTATUS_FIELDS_TO_HIDE`` to hide unwanted fields from ``{gitstatus}`` prompt field.
* Added number of lines added and removed to gitstatus
* Saving current working directory (cwd) to the history.
* Added XONSH_HISTORY_SAVE_CWD environment variable.
* Added environment variable ``$COMPLETE_DOTS`` to specify how current and previous directories should be tab completed in cd ('./', '../'):
- ``always`` Always complete paths with ./ and ../
- ``never`` Never complete paths with ./ and ../
- ``matching`` Complete if path starts with . or ..
* Complete ``import`` keyword in ``from ... import`` statements.
* Enabled case-insensitive completions for the ``jedi`` xontrib.
* Non-exclusive completers that enable aggregating multiple completer results.
* New ``$XONSH_CAPTURE_ALWAYS`` variable for opt-in interactive capturing.
Since this capturing breaks background jobs and some interactive programs (like ``git`` invoking an editor),
This behavior is now opt-in using this variable.
See https://github.com/xonsh/xonsh/pull/4283 and linked issues.
* Wrap selection with quote/parens when ``$XONSH_AUTOPAIR=True``.
* Now xonsh will work with Python 3.10. (Match statement is not supported).
* In addition to reading single rc files at startup (``/etc/xonshrc``, ``~/.config/xonsh/rc.xsh``),
xonsh now also supports rc.d-style config directories, from which all files are sourced. This is
designed to support drop-in style configuration where you could, for example, have a common config
file shared across multiple machines and a separate machine specific file.

This is controlled by the environment variable ``XONSHRC_DIR``, which defaults to
``["/etc/xonsh/rc.d", "~/.config/xonsh/rc.d"]``. If those directories exist, then any ``xsh`` files
contained within are sorted and then sourced.
* Added xontrib-prompt-starship - Starship prompt in xonsh shell.
* Added XONSH_SUBPROC_CAPTURED_PRINT_STDERR (default False) environment variable to hide unwanted printing the stderr when using captured object.
* A ``$XONSH_TRACE_COMPLETIONS`` variable for completions debugging.
* Added warning about prompt-toolkit in the welcome message.
* Added history backend name to the xonfig.
* `xontrib-linuxbrew <https://github.com/eugenesvk/xontrib-linuxbrew>`_ to add Homebrew's shell environment to xonsh shell on Linux
* Added xontrib-macro-lib - the library of the useful macros for the xonsh shell: https://github.com/anki-code/xontrib-macro-lib

**Changed:**

* update imphooks encoding regex to match the newer version at PEP 263
* Enabled bracketed paste mode for readline to protect against paste jacking
* The group of environment variables around history moved to the "Interactive Prompt History" section.
* Disabled completing subpaths for commands in ``jedi``.
* Improved ``which`` output for non-simple aliases
* New json history will be in XONSH_DATA_DIR/history_json directory.
* Completers for ``and/or``, ``&&/||/|`` and environment variables are now non-exclusive.
* Disabled ptk copying words/lines to clipboard on deletion (can be re-enabled with ``$XONSH_COPY_ON_DELETE``).
* Separated between ``XONSH_DEBUG`` and ``XONSH_NO_AMALGAMATE``. Setting ``XONSH_DEBUG=1`` now acts like ``XONSH_DEBUG=2`` before (basic information like input transformation, command replacement) and ``XONSH_DEBUG=2`` like ``XONSH_DEBUG=1`` before (more debugging information presented, like PLY parsing messages).
* Cleaned up available aliases for ``shell_type``
* Speedup commands-cache by saving results between runs and use the last run's result
* The ``completer add`` command after the non-exclusive completers.
This means it will not block them from adding their completions.
* Updated the tab-completion tutorial.

**Fixed:**

* handle importing/decoding user modules with a 'UTF-8 with BOM' encoding (4160)
* Fixed XONSH_HISTORY_FILE that has the actual path from the history backend now
* Annotated assignments (``x: int = 42``, ``x: int``).
* Fixed xpip sudo behavior in xonsh AppImage.
* Prevent cancelled future errors for async prompt ($ENABLE_ASYNC_PROMPT) fields from bubbling up (and destroying the prompt's formatting)
* $() no longer silently captures stderr
* Added catching callable argument and raising appropriate exception
* Crashing command-not-found output for bad file names on linux.
* Fixed error message when an empty command is run
* Fixed $ crash when no output is sent out by the command
* Fixed xonsh crash when launched using `xonsh -c '$("")'`
* now abbrevs callback will not remove word from ``buffer.text``. See https://github.com/xonsh/xonsh/issues/3642#issuecomment-793789741
* Fixed the incorrect SyntaxError that was thrown when a subprocess command was preceded by a comment ending with a colon
* Fixed the missing auto-indentation in readline and prompt_toolkit when a statement ending with a colon was followed by a comment
* Fixed the incorrect auto-indentation in prompt_toolkit when a comment ended with a colon
* Fixed JSON history garbage collection for XONSH_HISTORY_SIZE in seconds.
* Fixed ``skip`` completer (completes ``sudo``, ``which`` and other commands).
* In a subprocess command, having whitespace in between the left bracket and the command no longer raises a SyntaxError.
* Reduced history reading when run script or command. Potential speed increasing.
* Fixed crash on statup if XONSH_COLOR_STYLE is set to something invalid.
* Fixed the colorize and/or keywords.
* Functions can be used for $TITLE, the same way as for $PROMPT. (4148)
* wsl detection works on archlinux wsl2 now (and hopefully everywhere)
* Fixed an exception when run xonfig wizard in no RC mode.
* Bash completions now handle quoted and space-containing arguments better.
* ``import`` completions always work.
* Test consistent RC loading behaviour in a variety of startup scenarios
* Absolute paths to executables don't break bash completions anymore
* Fix colors and text in the welcome message.

**Authors:**

* Gil Forsyth
* anki-code
* Noortheen Raja
* Gyuri Horak
* Daniel Shimon
* Matthias Bussonnier
* Gordon Ball
* cryzed
* Peter Ye
* Evgeny
* Jeremy Schlatter
* jmoranos
* Walter A. Boring IV
* bhawkins
* JackofSpades707
* Luiz Antonio Lazoti
* francium

====================

**Added:**

* Add new internal command "disown" to remove background jobs from the shell's job list
* Python3.9 issues with subscriptor forms fixed.
* added `xontrib-cd <https://github.com/eugenesvk/xontrib-cd>`_
* Added **xontrib-history-encrypt** - new history backend that encrypt the xonsh shell commands history to prevent leaking sensitive data. If you like the idea give a star to the repository https://github.com/anki-code/xontrib-history-encrypt

**Changed:**

* New awesome landing on https://xon.sh - feel free to share and tweet!
* History files (json, sqlite) now have 600 (rw only for user) permission by default.
* PTK(python-prompt-toolkit) is no longer vendored with xonsh.

**Fixed:**

* Fixed a bug where "cd" and "rmdir" would return non-directory completions
* SQLite History Backend: show message instead of exiting when disk is full.

**Authors:**

* Gil Forsyth
* anki-code
* Noortheen Raja
* Tejasvi S Tomar
* Evgeny
* Adam Schwalm
* Nate Simon