Scrapy-zyte-smartproxy

Latest version: v2.3.4

Safety actively analyzes 641872 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 2 of 3

2.1.0

Use a custom logger instead of the root one

2.0.0

Following the upstream rebranding of Crawlera as Zyte Smart Proxy Manager, `scrapy-crawlera` has been renamed as `scrapy-zyte-smartproxy`, with the following backward-incompatible changes:

- The repository name and Python Package Index (PyPI) name are now `scrapy-zyte-smartproxy`.
- Setting prefixes have switched from `CRAWLERA_` to `ZYTE_SMARTPROXY_`.
- Spider attribute prefixes and request meta key prefixes have switched from `crawlera_` to `zyte_smartproxy_`.
- `scrapy_crawlera` is now `scrapy_zyte_smartproxy`.
- `CrawleraMiddleware` is now `ZyteSmartProxyMiddleware`, and its default `url` is now `http://proxy.zyte.com:8011`.
- Stat prefixes have switched from `crawlera/` to `zyte_smartproxy/`.
- The online documentation is moving to https://scrapy-zyte-smartproxy.readthedocs.io/

Note: Zyte Smart Proxy Manager headers continue to use the `X-Crawlera-` prefix.
- In addition to that, the `X-Crawlera-Client` header is now automatically included in all requests.

1.7.0

- Added more stats to better understanding the internal states.
- Log warning when using `https://` protocol.
- Add default `http://` protocol in case of none provided, and log warning about it.
- Fix duplicated request when the response is not from crawlera, this was causing an infinite loop of retries when `dont_filter=True`.

1.6.0

- Enable crawlera on demand by setting ``CRAWLERA_FORCE_ENABLE_ON_HTTP_CODES``

1.5.1

- Remove username and password from settings since it's removed from crawlera.
- Include affected spider in logs.
- Handle situations when crawlera is restarted and reply with 407's for a few minutes
by retrying the requests with a exponential backoff system.

1.5.0

- Correctly check for bans in crawlera (Jobs will not get banned on non ban 503's).
- Exponential backoff when crawlera doesn't have proxies available.
- Fix `dont_proxy=False` header disabling crawlera when it is enabled.

Page 2 of 3

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.