Webchanges

Latest version: v3.26.0

Safety actively analyzes 688931 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 6 of 9

3.9

Not secure
===================
2022-01-26

Changed
-------
* The method ``bs4`` of filter ``html2text`` has a new ``strip`` sub-directive which is passed to BeautifulSoup, and
its default value has changed to false to conform to BeautifulSoup's default. This gives better output in most
cases. To restore the previous non-standard behavior, add the ``strip: true`` sub-directive to the ``html2text``
filter of jobs.
* Pyppeteer (used for ``url`` jobs with ``use_browser: true``) is now crashing during certain tests with Python 3.7.
There will be no new development to fix this as the use of Pyppeteer will soon be deprecated in favor of Playwright.
See above to start using Playwright now (highly suggested).

Added
-----
* The method ``bs4`` of filter ``html2text`` now accepts the sub-directives ``separator`` and ``strip``.
* When using the command line argument ``--test-diff``, the output can now be sent to a specific reporter by also
specifying the ``--test-reporter`` argument. For example, if running on a machine with a web browser, you can see
the HTML version of the last diff(s) from job 1 with ``webchanges --test-diff 1 --test-reporter browser`` on your
local browser.
* New filter ``remove-duplicate-lines``. Contributed by `Michael Sverdlin <https://github.com/sveder>`__ upstream `here
<https://github.com/thp/urlwatch/pull/653>`__ (with modifications).
* New filter ``csv2text``. Contributed by `Michael Sverdlin <https://github.com/sveder>`__ upstream `here
<https://github.com/thp/urlwatch/pull/658>`__ (with modifications).
* The ``html`` report type has a new job directive ``monospace`` which sets the output to use a monospace font.
This can be useful e.g. for tabular text extracted by the ``pdf2text`` filter.
* The ``command_run`` report type has a new environment variable ``WEBCHANGES_CHANGED_JOBS_JSON``.
* Opt-in to use Playwright for jobs with ``use_browser: true`` instead of pyppeteer (see above).

Fixed
-----
* During conversion of Markdown to HTML,
* Code blocks were not rendered without wrapping and in monospace font;
* Spaces immediately after `` (code block opening) were being dropped.
* The ``email`` reporter's ``sendmail`` sub-directive was not passing the ``from`` sub-directive (when specified) to
the ``sendmail`` executable as an ``-f`` command line argument. Contributed by
`Jonas Witschel <https://github.com/diabonas>`__ upstream `here <https://github.com/thp/urlwatch/pull/671>`__ (with
modifications).
* HTML characters were not being unescaped when the job name is determined from the <title> tag of the data monitored
(if present).
* Command line argument ``--test-diff`` was only showing the last diff instead of all saved ones.
* The ``command_run`` report type was not setting variables ``count`` and ``jobs`` (always 0). Contributed by
`Brian Rak <https://github.com/devicenull>`__ in `#23 <https://github.com/mborsetti/webchanges/issues/23>`__.

Documentation
-------------
* Updated the "recipe" for monitoring Facebook public posts.
* Improved documentation for filter ``pdf2text``.

Internals
---------
* Support for Python 3.10 (except for ``url`` jobs with ``use_browser`` using pyppeteer since it does not yet support
it; use Playwright instead).
* Improved speed of detection and handling of lines starting with spaces during conversion of Markdown to HTML.
* Logging (``--verbose``) now shows thread IDs to help with debugging.

Known issues
------------
* Pyppeteer (used for ``url`` jobs with ``use_browser: true``) is now crashing during certain tests with Python 3.7.
There will be no new development to fix this as the use of Pyppeteer will soon be deprecated in favor of Playwright.
See above to start using Playwright now (highly suggested).

3.8.3

Not secure
====================
2021-08-29

Fixed
-----
* Fixed incorrect handling of timeout when checking if new version has been released.

Internals
---------
* DictType hints for configuration.

3.8.2

Not secure
====================
2021-08-19

⚠ Breaking Changes (dependencies)
---------------------------------
* Filter ``pdf2text``'s dependency Python package `pdftotext <https://github.com/jalan/pdftotext>`__ in its latest
version 2.2.0 has changed the way it displays text to no longer try to emulate formatting (columns etc.). This is
generally a welcome improvement as changes in formatting no longer trigger change reports, but if you want to
return to the previous layout we have added a ``physical`` sub-directive which you need to set to ``true`` on the
jobs affected. **Note that otherwise all your** ``pdf2text`` **jobs will report changes (in formatting) the first
time they are run after the pdftotext Python package is updated**.

Changed
-------
* Updated default Chromium executables to revisions equivalent to Chromium 92.0.4515.131 (latest stable release); this
fixes unsupported browser error thrown by certain websites. Use ``webchanges --chromium-directory`` to locate where
older revision were downloaded to delete them manually.

Added
-----
* Filter ``pdf2text`` now supports the ``raw`` and ``physical`` sub-directives, which are passed to the underlying
Python package `pdftotext <https://github.com/jalan/pdftotext>`__ (version 2.2.0 or higher).
* New ``--chromium-directory`` command line displays the directory where the downloaded Chromium executables are
located to facilitate the deletion of older revisions.
* Footer now indicates if the run was made with a jobs file whose stem name is not the default 'jobs', to ease
identification when running *webchanges* with a variety of jobs files.

Fixed
-----
* Fixed legacy code handling ``--edit-config`` command line argument to allow editing of a configuration file
with YAML syntax errors (`15 <https://github.com/mborsetti/webchanges/issues/15>`__ by
`Markus Weimar <https://github.com/Markus00000>`__).
* Telegram reporter documentation was missing instructions on how to notify channels (`16
<https://github.com/mborsetti/webchanges/issues/16>`__ by `Sean Tauber <https://github.com/buzzeddesign>`__).

Internals
---------
* Type hints are checked during pre-commit by `mypy <http://www.mypy-lang.org/>`__.
* Imports are rearranged during pre-commit by `isort <https://pycqa.github.io/isort/>`__.
* Now testing all database engines, including redis, and more, adding 4 percentage points of code coverage to 81%.
* The name of a FilterBase subclass is always its __kind__ + Filter (e.g. the class for ``element-by-id`` filter is
named ElementByIDFilter and not GetElementByID)

3.8.1

Not secure
====================
2021-08-03

Fixed
-----
* Files in the new _vendored directory are now installed correctly.

3.8

Not secure
====================
2021-07-31

Added
-----
* ``url`` jobs with ``use_browser: true`` (i.e. using *Pyppeteer*) now recognize ``data`` and ``method`` directives,
enabling e.g. to make a ``POST`` HTTP request using a browser with JavaScript support.
* New ``tz`` key for ``report`` in the configuration sets the timezone for the diff in reports (useful if running
e.g. on a cloud server in a different timezone). See `documentation
<https://webchanges.readthedocs.io/en/stable/reporters.html#tz>`__.
* New ``run_command`` reporter to execute a command and pass the report text as its input. Suggested by `Marcos Alano
<https://github.com/mhalano>`__ upstream `here <https://github.com/thp/urlwatch/issues/650>`__.
* New ``remove_repeated`` filter to remove repeated lines (similar to Unix's ``uniq``). Suggested by `Michael
Sverdlin <https://github.com/Sveder>`__ upstream `here <https://github.com/thp/urlwatch/pull/653>`__.
* The ``user_visible_url`` job directive now applies to all type of jobs, including ``command`` ones. Suggested by
`kongomongo <https://github.com/kongomongo>`__ upstream `here <https://github.com/thp/urlwatch/issue/608>`__.
* The ``--delete-snapshot`` command line argument now works with Redis database engine (``--database-engine redis``).
Contributed by `Scott MacVicar <https://github.com/scottmac>`__ with pull request
`13 <https://github.com/mborsetti/webchanges/pull/13>`__.
* The ``execute`` filter (and ``shellpipe``) sets more environment variables to allow for more flexibility; see improved
`documentation <https://webchanges.readthedocs.io/en/stable/filters.html#execute>`__ (including more examples).
* Negative job indices are allowed; for example, run ``webchanges -1`` to only run the last job of your jobs list, or
``webchanges --test -2`` to test the second to last job of your jobs list.
* Configuration file is now checked for invalid directives (e.g. typos) when program is run.
* Whenever a HTTP client error (4xx) response is received, in ``--verbose`` mode the content of the response is
displayed with the error.
* If a newer version of **webchanges** has been released to PyPI, an advisory notice is printed to stdout and
added to the report footer (if footer is enabled).

Fixed
-----
* The ``html2text`` filter's method ``strip_tags`` was returning HTML character references (e.g. &gt;, &62;, &x3e;)
instead of the corresponding Unicode characters.
* Fixed a rare case when html report would not correctly reconstruct a clickable link from Markdown for items inside
elements in a list.
* When using the ``--edit`` or ``--edit-config`` command line arguments to edit jobs or configuration files, symbolic
links are no longer overwritten. Reported by `snowman <https://github.com/snowman>`__ upstream
`here <https://github.com/thp/urlwatch/issues/604>`__.

Internals
---------
* ``--verbose`` command line argument will now list configuration keys 'missing' from the file, keys for which default
values have been used.
* ``tox`` testing can now be run in parallel using ``tox --parallel``.
* Additional testing, adding 3 percentage points of coverage to 78%.
* bump2version now follows `PEP440 <https://www.python.org/dev/peps/pep-0440/>`__ and has new documentation in
the file ``.bumpversion.txt`` (cannot document ``.bumpversion.cfg`` as remarks get deleted at every version bump).
* Added a vendored version of packaging.version.parse() from `Packaging <https://www.pypi.com/project/packaging/>`__
20.9, released on 2021-02-20, used to check if the version in PyPI is higher than the current one.
* Migrated from unmaintained Python package AppDirs to its friendly fork `platformdirs
<https://github.com/platformdirs/platformdirs>`__, which is maintained and offers more functionality. Unless used
by another package, you can uninstall appdirs with ``pip uninstall appdirs``.

3.7

====================
2021-06-27

⚠ Breaking Changes
------------------
* Removed Python 3.6 support to simplify code. Older Python versions are supported for 3 years after being obsoleted by
a new major release; as Python 3.7 was released on 27 June 2018, the last date of Python 3.6 support was 26 June 2021

Changed
-------
* Improved ``telegram`` reporter now uses MarkdownV2 and preserves most formatting of HTML sites processed by the
``html2text`` filter, e.g. clickable links, bolding, underlining, italics and strikethrough

Added
-----
* New filter ``execute`` to filter the data using an executable without invoking the shell (as ``shellpipe`` does)
and therefore exposing to additional security risks
* New sub-directive ``silent`` for ``telegram`` reporter to receive a notification with no sound (true/false) (default:
false)
* Github Issues templates for bug reports and feature requests

Fixed
-----
* Job ``headers`` stored in the configuration file (``config.yaml``) are now merged correctly and case-insensitively
with those present in the job (in ``jobs.yaml``). A header in the job replaces a header by the same name if already
present in the configuration file, otherwise is added to the ones present in the configuration file.
* Fixed ``TypeError: expected string or bytes-like object`` error in cookiejar (called by requests module) caused by
some ``cookies`` being read from the jobs YAML file in other formats

Internals
---------
* Strengthened security with `bandit <https://pypi.org/project/bandit/>`__ to catch common security issues
* Standardized code formatting with `black <https://pypi.org/project/black/>`__
* Improved pre-commit speed by using local libraries when practical
* More improvements to type hinting (moving towards testing with `mypy <https://pypi.org/project/mypy/>`__)
* Removed module jobs_browser.py (needed only for Python 3.6)

Page 6 of 9

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.