Wpull

==================

* Fixed: `KeyError` crash when psutil was not installed.
* Fixed: `AttributeError` proxy error using PhantomJS due to response body not written to a file.

================

* Removed: Lua scripting support and its Python counterpart (``--lua-script`` and ``--python-script``).
* Removed: Python 3.2 & 3.3 support.
* Removed: PyPy support.
* Changed: IP addresses are normalized to a standard notation to avoid fetching duplicates such as IPv4 addresses written in hexadecimal or long-hand IPv6 addresses.
* Changed: Scripting is now done using plugin interface via ``--plugin-script``.
* Fixed: Support for Python 3.5.
* Fixed: FTP unable to handle directory listing with date in MMM DD YYYY and filename containing YYYY-MM-DD text.
* Fixed: Downloads through the proxy (such as PhantomJS) now show up in the database and can be controlled through scripting.
* Fixed: `NotFound` error when converting links in CSS file that contain URLs that were not fetched.
* Fixed: When resuming a forcefully interrupted crawl (e.g., a crash) using a database, URLs in progress were not restarted because they were not reset in the database when the program started up.


Backwards incompatibility
+++++++++++++++++++++++++

This release contains backwards incompatible changes to the database
schema and scripting interface.

If you use ``--database``, the database created by older versions in
Wpull cannot be used in this version.

Scripting hook code will need to be rewritten to use the new API. See
the new documentation for scripting for the new style of interfacing
with Wpull.

Additionally for scripts, the internal event loop has switched from
Trollius to built-in Asyncio.

==================

* Removed: cx_freeze build support.
* Deprecated: Lua Scripting support will be removed in next release.
* Deprecated: Python 3.2 & 3.3 support will be removed in the next release.
* Deprecated: PyPy support will be removed in the next release.
* Fixed: Error when logging in with FTP to servers that don't need a password.
* Fixed: ValueError when downloading URLs that contain unencoded unprintable characters like Zero Width Non-Joiner or Right to Left Mark.

==================

* Fixed: ``--output-document`` file doesn't contain content.
* Fixed: OverflowError when URL contains invalid port number greater than 65535 or less than 0.
* Fixed: AssertionError when saving IPv4-mapped IPv6 addresses to WARC files.
* Fixed: AttributeError when running with installed Trollius 2.0.
* Changed: The setup file no longer requires optional psutil.

==================

* Fixed: OverflowError with URLs with large port numbers.
* Fixed: TypeError when using standard input as input file (``--input-file -``).
* Changed: using ``--youtube-dl`` respects ``--inet4-only`` and ``--no-check-certificate`` now.

================

* Fixed: Connecting to sites with IPv4 & IPv6 support resulted in errors when IPv6 was not supported by the local network. Connections now use Happy Eyeballs Algorithm for IPv4 & IPv6 dual-stack support.
* Fixed: SQLAlchemy error with PyPy and SQLAlchemy 1.0.
* Fixed: Input URLs are not fetched in order. Regression since 1.1.
* Fixed: UnicodeEncodeError when fetching FTP files with non-ASCII filenames.
* Fixed: Session cookies not loaded when using ``--load-cookies``.
* Fixed: ``--keep-session-cookies`` was always on.
* Changed: FTP communication uses UTF-8 instead of Latin-1.
* Changed: ``--prefer-family=none`` is now default.
* Added: ``none`` as a choice to ``--prefer-family``.
* Added: ``--no-glob`` and FTP filename glob support.