Zyte-spider-templates

------------------

* Added new input parameters:

* ``urls`` accepts a newline-delimited list of URLs.

* ``urls_file`` accepts a URL that points to a plain-text file with a
newline-delimited list of URLs.

Only one of ``url``, ``urls`` and ``urls_file`` should be used at a time.

* Added new crawling strategies:

* ``automatic`` - uses heuristics to see if an input URL is a homepage, for
which it uses a modified ``full`` strategy where other links are discovered
only in the homepage. Otherwise, it assumes it's a navigation page and uses
the existing ``navigation`` strategy.

* ``direct_item`` - input URLs are directly extracted as products.

* Added new parameters classes: ``LocationParam`` and ``PostalAddress``. Note
that these are available for use when customizing the templates and are not
currently being utilized by any template.

* Backward incompatible changes:

* ``automatic`` becomes the new default crawling strategy instead of ``full``.

* CI test improvements.

------------------

* Implemented :ref:`mixin classes for spider parameters <parameter-mixins>`, to
improve reuse.

* Improved docs, providing an example about overriding existing parameters when
:ref:`customizing parameters <custom-params>`, and featuring
:class:`~web_poet.AnyResponse` in the :ref:`example about overriding parsing
<override-parsing>`.

------------------

* The
:class:`~zyte_spider_templates.spiders.ecommerce.EcommerceSpiderParams.crawl_strategy`
parameter of
:class:`~zyte_spider_templates.spiders.ecommerce.EcommerceSpider`
now defaults to
:attr:`~zyte_spider_templates.spiders.ecommerce.EcommerceCrawlStrategy.full`
instead of
:attr:`~zyte_spider_templates.spiders.ecommerce.EcommerceCrawlStrategy.navigation`.
We also reworded some descriptions of :enum:`~.EcommerceCrawlStrategy` values
for clarification.

------------------

* Updated requirement versions:

* :doc:`scrapy-poet <scrapy-poet:index>` >= 0.21.0
* :doc:`scrapy-zyte-api <scrapy-zyte-api:index>` >= 0.16.0

* With the updated dependencies above, this fixes the issue of having 2 separate
Zyte API Requests (*productNavigation* and *httpResponseBody*) for the same URL. Note
that this issue only occurs when requesting product navigation pages.

* Moved :class:`zyte_spider_templates.spiders.ecommerce.ExtractFrom` into
:class:`zyte_spider_templates.spiders.base.ExtractFrom`.

------------------

* Improved the :attr:`zyte_spider_templates.spiders.base.BaseSpiderParams.url`
description.

------------------

* Fixed the ``extract_from`` spider parameter that wasn't working.

* The *"www."* prefix is now removed when setting the spider's
:attr:`~scrapy.Spider.allowed_domains`.

* The :attr:`zyte_common_items.ProductNavigation.nextPage` link won't be crawled
if :attr:`zyte_common_items.ProductNavigation.items` is empty.

* :class:`zyte_common_items.Product` items that are dropped due to low probability
*(below 0.1)* are now logged in stats: ``drop_item/product/low_probability``.

* :class:`zyte_spider_templates.pages.HeuristicsProductNavigationPage` now
inherits from :class:`zyte_common_items.AutoProductNavigationPage` instead of
:class:`zyte_common_items.BaseProductNavigationPage`.

* Moved e-commerce code from :class:`zyte_spider_templates.spiders.base.BaseSpider`
to :class:`zyte_spider_templates.spiders.ecommerce.EcommerceSpider`.

* Documentation improvements.