Types-lxml

Breaking changes
===========

- `Mypy` 1.9 is required, dropping 1.5 support. 1.6 - 1.8 was never supported.
- `lxml.ElementInclude` completely reworked

Features
=====

- PEP 696 support, simplifying usage of some subscripted types (42)
- As a convenient side effect, `lxml.html` parser constructor signatures can be removed
- All annotations do provide default values in their signatures now instead of `...`

Bug fix and small changes
================

- Type of `_Comment.text` property (and those of similar elements) is always `str` (46, thanks to eemeli)
- Tag selector argument in element iterator methods should support keyword with a single tag (45, thanks to eemeli)
- `html.fragments_fromstring()` should receive same fix as `html.html5parser.fragments_fromstring()` do (43, thanks to Wuestengecko)
- `overload` for `etree.SubElement()` on handling of `HtmlElement` and `ObjectifiedElement`
- Some exported constants were missing from `lxml.ElementInclude` stub
- `html.soupparser` module functions return type depends on `makeelement` argument
- Keyword arguments in `html.soupparser` module functions are explicitly listed now (instead of generic `**kwargs` before)
- The 2 arguments in `html.diff.html_annotate()` should align their annotation types
- `html.submit_form()` return type depends on the result of `open_http` function argument
- Add missing exported variable for `lxml.isoschematron`
- Uppercase variants of output method arguments ("HTML", "TEXT", "XML") were dropped

Internal changes
==========

- Usual runtime test additions: `lxml.html.soupparser`, `lxml.ElementInclude`, various exported constants
- Runtime tests also do test against lxml 5.2

Breaking change

- Requires `cssselect ⩾ 1.2` for annotation in `lxml.cssselect`, since `cssselect` is now inline annotated.

Bug fix and small changes

- Compatibility with `pyright ⩾ 1.1.353`
- In `etree.clean_*` functions, first argument (the Element or ElementTree to be processed) must be strictly positional
- `etree._LogEntry.filename` property is never empty, as it uses the value `<string>` as fallback
- `etree._BaseErrorLog.receive()` argument name was wrong
- Self brewed `SupportsReadClose` protocol dropped, replacing with more standardized `SupportsRead`
- `html.html5parser.parse()` should support data stream as input
- `html.html5parser.fragments_fromstring()` return type is dependent on `no_leading_text` argument
- `encoding` arguments in various methods / functions used to only support ASCII and UTF-8 as byte encodings, now the restriction is lifted
- Place some `typing` usage under python version check (`if sys.version_info >= (3, x)`)
- `etree.PyErrorLog` constructor shouldn't accept 2 logger arguments simultaneously
- `etree.PyErrorLog.level_map` property reverted to vanilla type (`int`) instead of our fake `enum`

Internal changes

- Some runtime tests are lxml version dependent ([34](https://github.com/abelcheung/types-lxml/issues/34), thanks to fabaff)
- Adds stub check for `_Element`, `_Comment` and `_ElementTree` ([33](https://github.com/abelcheung/types-lxml/issues/33), thanks to udifuchs)
- Following stub tests migrated to runtime: `_Attrib`, `_ErrorLog` and friends, `html5lib`

Bug fix and small changes

- Add back `HtmlProcessingInstruction` element (28, thanks to eliotwrobson)
- Silence `pyright` ⩾ 1.1.345 warning on overriding read-write property with read-only one (`ObjectifyElement.text`)

Documentation

- `mypy` ⩾ 1.6 does not support PEP702, thus shouldn't be used with `types-lxml`

Internal changes

- Stub test suite uses `mypy` 1.5.x now

Bug Fix

- Types for emitted events and values in `iterparse()` were not optimal (issue 19, thanks to Daverball)
- Most `html` link and clean functions should be unable to process `ElementTree`, except `Cleaner.clean_html()`

Feature

- Completed following modules, thus _really_ having `lxml` fully covered (sans a few submodules that will never be implemented):
- `lxml.html.diff`
- `lxml.ElementInclude`
- Declares support for Python 3.12
- Update for upcoming `lxml` 5.0
- `Schematron` constructor arguments
- Some obsolete functions removed

Internal change

- Start implementing runtime type checks and compare with static type checker results, utilizing [`typeguard`](https://github.com/agronholm/typeguard) and [`pyright`](https://github.com/microsoft/pyright)
- Use [`setuptools_scm`](https://github.com/pypa/setuptools_scm) in place of [`pdm-backend`](https://github.com/pdm-project/pdm-backend) as package build backend

----

Here is the [list of change since last release](https://github.com/abelcheung/types-lxml/compare/2023.3.28...2023.10.21). Besides, please check out [release notes for previous release](https://github.com/abelcheung/types-lxml/releases/tag/2023.3.28) as well, since it contains substantial changes.

The list of changes [since last release](https://github.com/abelcheung/types-lxml/compare/2023.02.11...2023.3.28) is huge, be it visible by users or not.

Breaking changes

- Class inheritance of `html.HtmlComment` and friends have changed to deviate from source code. Now they are 'thought' to inherit from `html.HtmlElement` within stubs, like the XML `etree._Element` counterpart. [Refer to wiki document](https://github.com/abelcheung/types-lxml/wiki/Element-inheritance-change) on how and why this change is done.
- Shelved custom parser target support (custom parser target is used when initiating XML / HTML parsers with `target=` argument), as current python typing system is deemed insufficient to get it working without plugins.
- Stub package only depends on other stub packages, following behavior of typeshed distributed stubs. This means `lxml` is no longer pulled in when installing `types-lxml`.
- `etree.SmartStr` reverted back to [its original class name](https://github.com/abelcheung/types-lxml/wiki/Smart-string-usage)
- `etree._ErrorLog` is now made a function that generates `etree._ListErrorLog` (despite the fact that it is a class in source code), according to actual created instance type

Significant changes / completion

- Completed following submodules and parts, thus removing [the partial status](https://peps.python.org/pep-0561/#partial-stub-packages) of `types-lxml` package:
- `lxml.etree` proper:
- [x] XSLT related classes / functions
- [x] XML:ID support
- [x] External [document and URI resolving](https://lxml.de/resolvers.html)
- [x] XInclude support
- [x] XPath and XSLT [extension function registry](https://lxml.de/extensions.html)
- [x] Error log and reporting, along with numerous bug fixes
- [x] `etree.iterparse` and `etree.iterwalk`
- [x] Various `ElementClassLookup` types
- [x] `lxml.objectify`
- [x] Includes all `DataElement` subtypes and type annotation support
- [x] `lxml.isoschematron`
- When subclassing XML elements, now most of its methods can be inherited without overriding output element type.

Smaller changes
- More extensive usage of Python 3.9-3.11 typing features, this is possible since `types-lxml` is external stub package and doesn't affect source code. Such as:
- Marking string constants as `LiteralString` ([PEP 675](https://peps.python.org/pep-0675/))
- Make type aliases more explicit ([PEP 613](https://peps.python.org/pep-0613/))
- Convenient `Self` when declaring methods ([PEP 673](https://peps.python.org/pep-0673/))
- Both `mypy` and `pyright` type checkers have strict mode turned on when verifying stub source
- `_Element.sourceline` property becomes read-only
- Re-added most deprecated methods in various places, with help from provisional [PEP 702](https://peps.python.org/pep-0702/) support (`deprecated`) in `pyright`
- Incorporate more docstring from official `lxml` classes, in case IDEs can display them in user interface.
- Force `_XPathEvaluatorBase` subclasses to make `__call__` available, by explicitly declaring it as abstract method within `_XPathEvaluatorBase`
- Removal of `http.open_http_urllib`, which is only intended as a fallback callback function for `html.submit_form()` without user intervention
- `libxml2` error constants become integer `enum` in stub
- Warn userland usage of dummy `etree.PyErrorLog.copy()`, because it is only intended for smoother internal `lxml` error handling.

Bug fixes
- File reading source (used in `file=` argument in `parse()` and friends) requirement relaxed
- `html.(X)HtmlParser` `__init__` was missing some arguments
- Convert `iter*` methods of Elements and some tag cleanup functions into `overload`, to better reflect its original intended arguments usage
- `etree.ElementBase` and similar public base element classes lacked `__init__`
- Setting of `etree.DocInfo` text properties now accepts `bytes`
- `name=` argument of `html.HtmlElementClassLookup()` doesn't accept `None`
- Concerning `_Comment`, `_Entity`, `_ProcessingInstruction`, and their subclasses
- `.tag` attribute now returns correct value (the basic etree element factory function)
- Users will be warned if they use these elements like normal XML `_Element` do, such as treating them as parent elements and insert children element into them

User visible change since previous release:

- Add types for [XML canonicalization function/class](https://lxml.de/api.html#serialisation) and [incremental generation context managers](https://lxml.de/api.html#incremental-xml-generation)

[Full changelog here.](../../compare/2022.11.8...2023.02.11)