Range-streams

Latest version: v1.3.0

Safety actively analyzes 682387 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

1.3.0

- :bug: 571433a Thanks to generalmimon for reporting (closes 41)

1.2.0

- :rocket: Async now supported for streams in single request mode
- `RangeStream` and its subclasses now have a `make_async_fetcher` method which will return an `AsyncFetcher` that you can pass a list of URLs to (optionally: a client to use), and an async callback function (which will receive: the `AsyncFetcher`, the `RangeStream` sub/class (with the received response), and the source URL which was requested)

Example showing the speedup this brings to PNGs, due to them being a very linear (unlike e.g. zip files) and [many-chunked](https://en.wikipedia.org/wiki/Portable_Network_Graphics#%22Chunks%22_within_the_file) format which leads to a large number of requests which annihilate non-async performance:

py
import range_streams
from range_streams.codecs.png import PngStream
from some_urls import urls
import httpx
from tqdm import tqdm
from time import time
from random import shuffle

async def callback(fetcher, stream, url):
await stream.enumerate_chunks_async()

sample_size = 60
dbl_smpl_sz = 2 * sample_size
print(f"Sampling {sample_size} of {len(urls)} PNG URLs each time (200px thumbnails)\n")

for i in range(4):
urls_sample = [u for u in urls[:dbl_smpl_sz]]
shuffle(urls_sample) Split a random sample
urls_a, urls_b = urls_sample[0:sample_size], urls_sample[sample_size:dbl_smpl_sz]

print("Synchronous")
t0 = time()
c = httpx.Client()
for u in tqdm(urls_a):
s = PngStream(url=u, client=c, enumerate_chunks=True, scan_ihdr=False)
del s
t1 = time()
print(f"Done in {t1-t0}s")

print("Asynchronous")
fetched = PngStream.make_async_fetcher(urls=urls_b, callback=callback)
fetched.make_calls()
t2 = time()
print(f"Done in {t2-t1}s")
print()


![async-range-streams-speed-test](https://user-images.githubusercontent.com/2979452/129285883-e58568e7-712a-44dd-84f5-70e7dda4737e.png)

1.0.0

- :mag: Windowed ranges
- :no_entry_sign: HEAD requests replaced with open-ended partial content requests
- :zap: Single request mode with the `single_request` flag (default `False` for all `RangeStream` sub/classes except the PNG codec `PngStream`)
- :bug: Bug fixes and more tests to stop them coming back!

0.4.0

0.1.0

- Efficient and lightweight range streaming with file-like objects
- 3 pruning levels when handling range overlaps
1) "replant": existing ranges that overlap a newly requested range will be re-requested or have their iterator truncated
2) "burn": existing ranges that overlap a newly requested range will be deleted
3) "strict": existing ranges that overlap a newly requested range will raise errors
- [98% test coverage](https://codecov.io/github/lmmx/range-streams)
- New design docs
- Fully type checked

- :package: PyPI [v0.1.0](https://pypi.org/project/range-streams/0.1.0/)

Links

Releases

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.