Scrapy-zyte-api

-------------------

* The new ``_ZYTE_API_USER_AGENT`` setting allows customizing the user agent
string reported to Zyte API.

Note that this setting is only meant for libraries and frameworks built on
top of scrapy-zyte-api, to report themselves to Zyte API, for client software
tracking and monitoring purposes. The value of this setting is *not* the
``User-Agent`` header sent to upstream websites when using Zyte API.

-------------------

* A new ``ZYTE_API_PROVIDER_PARAMS`` setting allows setting Zyte API
parameters, like ``geolocation``, to be included in all Zyte API requests by
the scrapy-poet provider.

* A new ``scrapy-zyte-api/request_args/<parameter>`` stat, counts the number of
requests containing a given Zyte API request parameter. For example,
``scrapy-zyte-api/request_args/url`` counts the number of Zyte API requests
with the URL parameter set (which should be all of them).

Experimental is treated as a namespace, and its parameters are the ones
counted, i.e. there is no ``scrapy-zyte-api/request_args/experimental`` stat,
but there are stats like
``scrapy-zyte-api/request_args/experimental.responseCookies``.

-------------------

* scrapy-zyte-api 0.11.0 accidentally increased the minimum required version of
scrapy-poet from 0.10.0 to 0.11.0. We have reverted that change and
implemented measures to prevent similar accidents in the future.

* Automatic parameter mapping no longer warns about dropping the
``Accept-Encoding`` header when the header value matches the Scrapy default.

* The README now mentions additional changes that may be necessary when
switching Twisted reactors on existing projects.

* The README now explains how status codes, from Zyte API or from wrapped
responses, are reflected in Scrapy stats.

-------------------

* Added a ``ZYTE_API_MAX_REQUESTS`` setting to limit the number of successful
Zyte API requests that a spider can send. Reaching the limit stops the
spider.

* Setting ``requestCookies`` to ``[]`` in the ``zyte_api_automap`` request
metadata field now triggers a warning.

-------------------

* Added more data types to the scrapy-poet provider:

* ``zyte_common_items.ProductList``
* ``zyte_common_items.ProductNavigation``
* ``zyte_common_items.Article``
* ``zyte_common_items.ArticleList``
* ``zyte_common_items.ArticleNavigation``

* Moved the new dependencies added in 0.9.0 and needed only for the scrapy-poet
provider (``scrapy-poet``, ``web-poet``, ``zyte-common-items``) into the new
optional feature ``[provider]``.

* Improved result caching in the scrapy-poet provider.

* Added a new setting, ``ZYTE_API_USE_ENV_PROXY``, which can be set to ``True``
to access Zyte API using a proxy configured in the local environment.

* Fixed getting the Scrapy Cloud job ID.

* Improved the documentation.

* Improved the CI configuration.

------------------

* New and updated requirements:

* packaging >= 20.0
* scrapy-poet >= 0.9.0
* web-poet >= 0.13.0
* zyte-common-items

* Added a scrapy-poet provider for Zyte API. Currently supported data types:

* ``web_poet.BrowserHtml``
* ``web_poet.BrowserResponse``
* ``zyte_common_items.Product``

* Added a ``zyte_api_default_params`` request meta key which allows users to
ignore the ``ZYTE_API_DEFAULT_PARAMS`` setting for individual requests.

* CI fixes.