Zyte-spider-templates

Latest version: v0.9.0

Safety actively analyzes 681866 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 2 of 2

0.6.0

------------------

* Fixed the ``extract_from`` spider parameter that wasn't working.

* The *"www."* prefix is now removed when setting the spider's
:attr:`~scrapy.Spider.allowed_domains`.

* The :attr:`zyte_common_items.ProductNavigation.nextPage` link won't be crawled
if :attr:`zyte_common_items.ProductNavigation.items` is empty.

* :class:`zyte_common_items.Product` items that are dropped due to low probability
*(below 0.1)* are now logged in stats: ``drop_item/product/low_probability``.

* :class:`zyte_spider_templates.pages.HeuristicsProductNavigationPage` now
inherits from :class:`zyte_common_items.AutoProductNavigationPage` instead of
:class:`zyte_common_items.BaseProductNavigationPage`.

* Moved e-commerce code from :class:`zyte_spider_templates.spiders.base.BaseSpider`
to :class:`zyte_spider_templates.spiders.ecommerce.EcommerceSpider`.

* Documentation improvements.

0.5.0

------------------

* The ``zyte_spider_templates.page_objects`` module is now deprecated in favor
of ``zyte_spider_templates.pages``, in line with ``web_poet.pages``.

0.4.0

------------------

* Products outside of the target domain can now be crawled using
:class:`zyte_spider_templates.middlewares.AllowOffsiteMiddleware`.

* Updated the documentation to also set up ``zyte_common_items.ZyteItemAdapter``.

* The ``max_requests`` spider parameter has now a default value of 100. Previously,
it was ``None`` which was unlimited.

* Improved the description of the ``max_requests`` spider parameter.

* Official support for Python 3.12.

* Misc documentation improvements.

0.3.0

------------------

* Added documentation.

* Added a middleware that logs information about the crawl in JSON format,
:class:`zyte_spider_templates.middlewares.CrawlingLogsMiddleware`. This
replaces the old crawling information that was difficult to parse using
regular expressions.

0.2.0

------------------

* Now requires ``zyte-common-items >= 0.12.0``.

* Added a new crawl strategy, "Pagination Only".

* Improved the request priority calculation based on the metadata probability
value.

* CI improvements.

0.1.0

------------------

Initial release.

Page 2 of 2

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.