Minference

Latest version: v0.1.5.post1

The latest version of minference with no known security vulnerabilities is 0.1.5.post1. We recommend installing version 0.1.5.post1.

The information on this page was curated by experts in our Cybersecurity Intelligence Team.

Latest release: v0.1.5.post1 at Aug. 13, 2024
License: MIT (MIT License)

Description

To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

Resources

GitHub
PyPi

Vulnerabilities

See all vulnerabilities

No known vulnerabilities found

Versions (7)

See all versions

Has known vulnerabilities

0.1.5.post1
0.1.5
0.1.4.post4
0.1.4.post3
0.1.4.post2
0.1.4.post1
0.1.4