Pywikibot

-----
*21 May 2022*

* Add support for kcgwiki (:phab:`T305282`)
* Raise InvalidTitleError instead of unspecific ValueError in ProofreadPage (:phab:`T308016`)
* Preload pages if GeneratorFactory.articlenotfilter_list is not empty; also set attribute ``is_preloading``.
* ClaimCollection.toJSON() should not ignore new claim (:phab:`T308245`)
* use linktrail via siteinfo and remove `update_linkrtrails` maintenance script
* Print counter statistic for all counters (:phab:`T307834`)
* Use proofreadpagesinindex query module
* Prioritize -namespaces options in `pagegenerators.handle_args` (:phab:`T222519`)
* Remove `ThreadList.stop_all()` method (:phab:`T307830`)
* L10N updates
* Improve get_charset_from_content_type function (:phab:`T307760`)
* A tiny cache wrapper was added to hold results of parameterless methods and properties
* Increase workers in preload_sites.py
* Close logging handlers before deleting them (:phab:`T91375`, :phab:`T286127`)
* Clear _sites cache if called with pwb wrapper (:phab:`T225594`)
* Enable short creation of a site if family name is equal to site code
* Use `exc_info=True` with pywikibot.exception() by default (:phab:`T306762`)
* Make IndexPage more robust when getting links in Page ns (:phab:`T307280`)
* Do not print log header twice in log files (:phab:`T264235`)
* Do not delegate logging output to the root logger (:phab:`T281643`)
* Add `get_charset_from_content_type` to extract the charset from the content-type response header

-----
*26 April 2022*

* Make logging system consistent, add pywikibot.info() alias for pywikibot.output() (:phab:`T85620`)
* L10N updates
* Circumvent circular import in tools module (:phab:`T306760`)
* Don't fix html inside syntaxhighlight parts in fixes.py (:phab:`T306723`)
* Make layer parameter optional in :func:`pywikibot.debug()<pywikibot.logging.debug>` (:phab:`T85620`)
* Retry for internal_api_error_DBQueryTimeoutError errors due to :phab:`T297708`
* Handle ParserError within xmlreader.XmlDump.parse() instead of raising an exception (:phab:`T306134`)
* XMLDumpOldPageGenerator is deprecated in favour of a `content` parameter (:phab:`T306134`)
* `use_disambig` BaseBot attribute was added to hande disambig skipping
* Deprecate RedirectPageBot and NoRedirectPageBot in favour of `use_redirects` attribute
* tools.formatter.color_format is deprecated and will be removed
* A new and easier color format was implemented; colors can be used like:
``'this is a <<green>>colored<<default>> text'``
* Unused and unsupported `xmlreader.XmlParserThread` was removed
* Use upercased IP user titles (:phab:`T306291`)
* Use pathlib to extract filename and file_package in pwb.py
* Fix isbn messages in fixes.py (:phab:`T306166`)
* Fix Page.revisions() with starttime (:phab:`T109181`)
* Use stream_output for messages inside input_list_choice method (:phab:`T305940`)
* Expand simulate query result (:phab:`T305918`)
* Do not delete text when updating a Revision (:phab:`T304786`)
* Re-enable scripts package version check with pwb wrapper (:phab:`T305799`)
* Provide textlib.ignore_case() as a public method
* Don't try to upcast timestamp from global userinfo if global account does not exists (:phab:`T305351`)
* Archived scripts were removed; create a Phabricator task to restore some (:phab:`T223826`)
* Add Lexeme support for Lexicographical data (:phab:`T189321`, :phab:`T305297`)
* enable all parameters of `APISite.imageusage()` with `FilePage.using_pages()`
* Don't raise `NoPageError` with `file_is_shared` (:phab:`T305182`)
* Fix URL of GoogleOCR
* Handle ratelimit with purgepages() (:phab:`T152597`)
* Add movesubpages parameter to Page.move() and APISite.movepage() (:phab:`T57084`)
* Do not iterate over sys.modules (:phab:`T304785`)

-----
*26 March 2022*

* Add FilePage.file_is_used property to determine whether a file is used on a site
* Add support for guwwiki and shnwikivoyage (:phab:`T303762`, :phab:`T302799`)
* TextExtracts support was aded (:phab:`T72682`)
* Unused `get_redirect` parameter of Page.getOldVersion() has been dropped
* Provide BasePage.get_parsed_page() as a public method
* Provide BuiltinNamespace.canonical_namespaces() with BuiltinNamespace IntEnum
* BuiltinNamespace got a canonical() method
* Enable nested templates with MultiTemplateMatchBuilder (:phab:`T110529`)
* Introduce APISite.simple_request as a public method
* Provide an Uploader class to upload files
* Enable use of deletetalk parameter of the delete API
* Fix contextlib redirection for terminal interfaces (:phab:`T283808`)
* No longer use win32_unicode for Python 3.6+ (:phab:`T281042`, :phab:`T283808`, :phab:`T303373`)
* L10N updates
* -cosmetic_changes (-cc) option allows to assign the value directly instead of toggle it
* distutils.util.strtobool() was implemented as tools.strtobool() due to :pep:`632`
* The "in" operator always return whether the siteinfo contains the key even it is not cached (:phab:`T302859`)
* Siteinfo.clear() and Siteinfo.is_cached() methods were added

-----
*26 February 2022*

**Improvements**

* i18n updates for date.py
* Add number transliteration of 'lo', 'ml', 'pa', 'te' to NON_LATIN_DIGITS
* Detect range blocks with Page.is_blocked() method (:phab:`T301282`)
* to_latin_digits() function was added to textlib as counterpart of to_local_digits() function
* api.Request.submit now handles search-title-disabled and search-text-disabled API Errors
* A show_diff parameter was added to Page.put() and Page.change_category()
* Allow categories when saving IndexPage (:phab:`T299806`)
* Add a new function case_escape to textlib
* Support inheritance of the __STATICREDIRECT__
* Avoid non-deteministic behavior in removeDisableParts
* Update isbn dependency and require python-stdnum >= 1.17
* Synchronize Page.linkedPages() parameters with Site.pagelinks() parameters
* Scripts hash bang was changed from python to python3
* i18n.bundles(), i18n.known_languages and i18n._get_bundle() functions were added
* Raise ConnectionError immediately if urllib3.NewConnectionError occurs (:phab:`T297994`, :phab:`T298859`)
* Make pywikibot messages available with site package (:phab:`T57109`, :phab:`T275981`)
* Add support for API:Redirects
* Enable shell script with Pywikibot site package
* Enable generate_user_files.py and generate_family_file with site-package (:phab:`T107629`)
* Add support for Python 3.11
* Pywikibot supports PyPy 3 (:phab:`T101592`)
* A new method User.is_locked() was added to determine whether the user is currently locked globally (:phab:`T249392`)
* A new method APISite.is_locked() was added to determine whether a given user or user id is locked globally
(:phab:`T249392`)
* APISite.get_globaluserinfo() method was added to retrieve globaluserinfo for any user or user id (:phab:`T163629`)
* APISite.globaluserinfo attribute may be deleted to force reload
* APISite.is_blocked() method has a force parameter to reload that info
* Allow family files in base_dir by default
* Make pwb wrapper script a pywikibot entry point for scripts (:phab:`T139143`, :phab:`T270480`)
* Enable -version and --version with pwb wrapper or code entry point (:phab:`T101828`)
* Add `title_delimiter_and_aliases` attribute to family files to support WikiHow family (:phab:`T294761`)
* BaseBot has a public collections.Counter for reading, writing and skipping a page
* Upload: Retry upload if 'copyuploadbaddomain' API error occurs (:phab:`T294825`)
* Update invisible characters from unicodedata 14.0.0
* Add support for Wikimedia OCR engine with proofreadpage
* Rewrite :func:`tools.itertools.intersect_generators` which makes it running up to 10'000 times faster.
(:phab:`T85623`, :phab:`T293276`)
* The cached output functionality from compat release was re-implemented
(:phab:`T151727`, :phab:`T73646`, :phab:`T74942`, :phab:`T132135`, :phab:`T144698`, :phab:`T196039`, :phab:`T280466`)
* L10N updates
* Adjust groupsize within pagegenerators.PreloadingGenerator (:phab:`T291770`)
* New "maxlimit" property was added to APISite (:phab:`T291770`)

**Bugfixes**

* Don't raise an exception if BlockEntry initializer found a hidden title (:phab:`T78152`)
* Fix KeyError in create_warnings_list (:phab:`T301610`)
* Enable similar script call of pwb.py on toolforge (:phab:`T298846`)
* Remove question mark character from forbidden file name characters (:phab:`T93482`)
* Enable -interwiki option with pagegenerators (:phab:`T57099`)
* Don't assert login result (:phab:`T298761`)
* Allow title placeholder $1 in the middle of an url (:phab:`T111513`, :phab:`T298078`)
* Don't create a Site object if pywikibot is not fully imported (:phab:`T298384`)
* Use page.site.data_repository when creating a _WbDataPage (:phab:`T296985`)
* Fix mysql AttributeError for sock.close() on toolforge (:phab:`T216741`)
* Only search user_script_paths inside config.base_dir (:phab:`T296204`)
* pywikibot.argv has been fixed for pwb.py wrapper if called with global args (:phab:`T254435`)
* Only ignore FileExistsError when creating the api cache (:phab:`T295924`)
* Only handle query limit if query module is limited (:phab:`T294836`)
* Upload: Only set filekey/offset for files with names (:phab:`T294916`)
* Make site parameter of textlib.replace_links() mandatory (:phab:`T294649`)
* Raise a generic ServerError if the http status code is unofficial (:phab:`T293208`)

**Breaking changes**

* Support of Python 3.5.0 - 3.5.2 has been dropped (:phab:`T286867`)
* generate_user_files.py, generate_user_files.py, shell.py and version.py were moved to pywikibot/scripts and must be
used with pwb wrapper script
* *See also Code cleanups below*

**Code cleanups**

* Deprecated http.get_fake_user_agent() function was removed
* FilePage.fileIsShared() was removed in favour of FilePage.file_is_shared()
* Page.canBeEdited() was removed in favour of Page.has_permission()
* BaseBot.stop() method were removed in favour of BaseBot.generator.close()
* showHelp() function was remove in favour of show_help
* CombinedPageGenerator pagegenerator was removed in favour of itertools.chain
* Remove deprecated echo.Notification.id
* Remove APISite.newfiles() method (:phab:`T168339`)
* Remove APISite.page_exists() method
* Raise a TypeError if BaseBot.init_page return None
* Remove private upload parameters in UploadRobot.upload_file(), FilePage.upload() and APISite.upload() methods
* Raise an Error exception if 'titles' is still used as where parameter in Site.search()
* Deprecated version.get_module_version() function was removed
* Deprecated setOptions/getOptions OptionHandler methods were removed
* Deprecated from_page() method of CosmeticChangesToolkit was removed
* Deprecated diff attribute of CosmeticChangesToolkit was removed in favour of show_diff
* Deprecated namespace and pageTitle parameter of CosmeticChangesToolkit were removed
* Remove deprecated BaseSite namespace shortcuts
* Remove deprecated Family.get_cr_templates method in favour of Site.category_redirects()
* Remove deprecated Page.put_async() method (:phab:`T193494`)
* Ignore baserevid parameter for several DataSite methods
* Remove deprecated preloaditempages method
* Remove disable_ssl_certificate_validation kwargs in http functions in favour of verify parameter (:phab:`T265206`)
* Deprecated PYWIKIBOT2 environment variables were removed
* version.ParseError was removed in favour of exceptions.VersionParseError
* specialbots.EditReplacement and specialbots.EditReplacementError were removed in favour of
exceptions.EditReplacementError
* site.PageInUse exception was removed in favour of exceptions.PageInUseError
* page.UnicodeToAsciiHtml and page.unicode2html were removed in favour of tools.chars.string_to_ascii_html and
tools.chars.string2html
* interwiki_graph.GraphImpossible and login.OAuthImpossible exception were removed in favour of ImportError
* i18n.TranslationError was removed in favour of exceptions.TranslationError
* WikiaFamily was removed in favour of FandomFamily
* data.api exceptions were removed in favour of exceptions module
* cosmetic_changes CANCEL_ALL/PAGE/METHOD/MATCH constants were removed in favour of CANCEL enum
* pywikibot.__release__ was removed in favour of pywikibot.__version__
* TextfilePageGenerator was replaced by TextIOPageGenerator
* PreloadingItemGenerator was replaced by PreloadingEntityGenerator
* DuplicateFilterPageGenerator was replaced by :func:`tools.itertools.filter_unique`
* ItemPage.concept_url method was replaced by ItemPage.concept_uri
* Outdated parameter names has been dropped
* Deprecated pywikibot.Error exception were removed in favour of pywikibot.exceptions.Error classes (:phab:`T280227`)
* Deprecated exception identifiers were removed (:phab:`T280227`)
* Deprecated date.FormatDate class was removed in favour of date.format_date function
* language_by_size property of wowwiki Family was removed in favour of codes attribute
* availableOptions was removed in favour of available_options
* config2 was removed in favour of config
* tools.RotatingFileHandler was removed in favour of logging.handlers.RotatingFileHandler
* tools.DotReadableDict, tools.LazyRegex and tools.DeprecatedRegex classes were removed
* tools.frozenmap was removed in favour of types.MappingProxyType
* tools.empty_iterator() was removed in favour of iter(())
* tools.concat_options() function was removed in favour of bot_choice.Option
* tools.is_IP was be removed in favour of tools.is_ip_address()
* textlib.unescape() function was be removed in favour of html.unescape()
* APISite.deletepage() and APISite.deleteoldimage() methods were removed in favour of APISite.delete()
* APISite.undeletepage() and APISite.undelete_file_versions() were be removed in favour of APISite.undelete() method

-----
*07 February 2022*

* L10N updates

-----
*27 January 2022*

* L10N updates