Sec-edgar-downloader

Fixed

- Anchor links inside of filings are now resolved correctly. Fragments and external links should now function as intended.
- Renamed `requirements.txt` to `requirements-dev.txt` in order to prevent confusion with the dependencies listed in `setup.py`.

New

- The `httpx` package has been replaced by `requests` to enable the use of an exponential backoff retry mechanism to help alleviate `403 Forbidden` errors some users are seeing. A request to `sec.gov` will be retried at most 10 times (with an exponential backoff applied to each request) before failing.
- A random `User-Agent` string is now included in the headers of each `GET` and `POST` request to `sec.gov`, rather than per session.

New

- HTTP connections are now re-used when possible (using [`httpx.Client()`](https://www.python-httpx.org/advanced/#client-instances)) to improve download performance.

Fixed

- Requests are now retried at most 5 times if a request fails. This should solve the `500 Server Error`s that some users are experiencing when downloading a large number of filings.

Changed

- Replaced the internal `requests` package with [`httpx`](https://github.com/encode/httpx), a more modern and performant alternative.

Fixed

- Fixed a `403 Client Error` that could randomly occur when bulk downloading a large number of filings. This error was most likely caused by recent changes to SEC rate-limiting behavior. It has been fixed by including a random user-agent string, generated by the [Faker package](https://faker.readthedocs.io/en/stable/providers/faker.providers.user_agent.html), in the request headers.

Fixed

- Fixed a `RecursionError` that could occur when downloading older filings with the `download_details` flag set to true. Thanks to neilbartlett for reporting and fixing this bug!

Fixed

- Downloads will no longer halt prematurely if a filing document (full or detail) cannot be found (e.g. when the EDGAR Search API outputs incorrect download URLs). Now, the package will automatically catch such network errors, print a helpful warning message, and then proceed to download the remaining filings.