Gptcache

🎉 Introduction to new functions of GPTCache

1. Support the temperature param

python
from gptcache.adapter import openai

openai.ChatCompletion.create(
model="gpt-3.5-turbo",
temperature = 1.0, Change temperature here
messages=[{
"role": "user",
"content": question
}],
)


2. Add the session layer

python
from gptcache.adapter import openai
from gptcache.session import Session

session = Session(name="my-session")
question = "what do you think about chatgpt"
openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "user", "content": question}
],
session=session
)


details: https://github.com/zilliztech/GPTCache/tree/main/examples#How-to-run-with-session

3. Support config cache with yaml for server

python
from gptcache.adapter.api import init_similar_cache_from_config

init_similar_cache_from_config(config_dir="cache_config_template.yml")


config file template: https://github.com/zilliztech/GPTCache/blob/main/cache_config_template.yml

4. Adapt the dolly model

python
from gptcache.adapter.dolly import Dolly

llm = Dolly.from_model(model="databricks/dolly-v2-3b")
llm(question)


What's Changed
* Use temperature to control possibility of skip_cache by jaelgu in https://github.com/zilliztech/GPTCache/pull/306
* Add template for similar cache init config by Chiiizzzy in https://github.com/zilliztech/GPTCache/pull/308
* Add dolly by junjiejiangjjj in https://github.com/zilliztech/GPTCache/pull/311
* Add session usage doc by shiyu22 in https://github.com/zilliztech/GPTCache/pull/310
* Add docs for temperature by jaelgu in https://github.com/zilliztech/GPTCache/pull/312
* Dolly and llama docs by junjiejiangjjj in https://github.com/zilliztech/GPTCache/pull/314
* Some minor polish on cache init by Chiiizzzy in https://github.com/zilliztech/GPTCache/pull/313
* Update the version to `0.1.21` by SimFG in https://github.com/zilliztech/GPTCache/pull/318


**Full Changelog**: https://github.com/zilliztech/GPTCache/compare/0.1.20...0.1.21

🎉 Introduction to new functions of GPTCache

1. support the `temperature` param, like openai

A non-negative number of sampling temperature, defaults to 0.
A higher temperature makes the output more random.
A lower temperature means a more deterministic and confident output.

2. Add llama adapter

python
from gptcache.adapter.llama_cpp import Llama

llm = Llama('./models/7B/ggml-model.bin')
answer = llm(prompt=question)


**Full Changelog**: https://github.com/zilliztech/GPTCache/compare/0.1.19...0.1.20

🎉 Introduction to new functions of GPTCache

1. Add stability sdk adapter (text -> image)

python
import os
import time

from gptcache import cache
from gptcache.processor.pre import get_prompt
from gptcache.adapter.stability_sdk import StabilityInference, generation
from gptcache.embedding import Onnx
from gptcache.manager.factory import manager_factory
from gptcache.similarity_evaluation.distance import SearchDistanceEvaluation

init gptcache
onnx = Onnx()
data_manager = manager_factory('sqlite,faiss,local',
data_dir='./',
vector_params={'dimension': onnx.dimension},
object_params={'path': './images'}
)
cache.init(
pre_embedding_func=get_prompt,
embedding_func=onnx.to_embeddings,
data_manager=data_manager,
similarity_evaluation=SearchDistanceEvaluation()
)

api_key = os.getenv('STABILITY_KEY', 'key-goes-here')

stability_api = StabilityInference(
key=os.environ['STABILITY_KEY'], API Key reference.
verbose=False, Print debug messages.
engine='stable-diffusion-xl-beta-v2-2-2', Set the engine to use for generation.
)

start = time.time()
answers = stability_api.generate(
prompt='a cat sitting besides a dog',
width=256,
height=256
)


stability reference: https://platform.stability.ai/docs/features/text-to-image

2. Add minigpt4 adapter

Notice: It cannot be used directly, it needs to cooperate with mini-GPT4 source code, refer to: https://github.com/Vision-CAIR/MiniGPT-4/pull/136

What's Changed
* Unify the format of `manager` variable names in `manager_factory` method by SimFG in https://github.com/zilliztech/GPTCache/pull/276
* Adapt stability_sdk by jaelgu in https://github.com/zilliztech/GPTCache/pull/277
* Add minigpt4 adapter by shiyu22 in https://github.com/zilliztech/GPTCache/pull/274
* Update docs by jaelgu in https://github.com/zilliztech/GPTCache/pull/278
* Make np evaluation positively correlated with the similarity. by wxywb in https://github.com/zilliztech/GPTCache/pull/280
* Add temperature_softmax in post processor by jaelgu in https://github.com/zilliztech/GPTCache/pull/282
* Update the version to `0.1.19` by SimFG in https://github.com/zilliztech/GPTCache/pull/283


**Full Changelog**: https://github.com/zilliztech/GPTCache/compare/0.1.18...0.1.19

🎉 Introduction to new functions of GPTCache

1. Add vqa bootcamp

reference: https://github.com/zilliztech/GPTCache/tree/main/docs/bootcamp/replicate

2. Add two streamlit multimodal demos

reference: https://github.com/zilliztech/GPTCache/tree/main/docs/bootcamp/streamlit

3. Add vit image embedding func

python
from gptcache.embedding import ViT

encoder = ViT(model="google/vit-base-patch16-384")
embed = encoder.to_embeddings(image)


4. Add `init_similar_cache` func for the GPTCache api module

python
from gptcache.adapter.api import init_similar_cache

init_similar_cache("cache_data")


5. The simple GPTCache server provides similar cache

- clone the GPTCache repo, `git clone https://github.com/zilliztech/GPTCache.git`
- install the gptcache model, `pip install gptcache`
- run the GPTCache server, `cd gptcache_server && python server.py`

What's Changed
* Add vqa bootcamp by shiyu22 in https://github.com/zilliztech/GPTCache/pull/263
* Update vqa bootcamp by shiyu22 in https://github.com/zilliztech/GPTCache/pull/265
* Modify np evaluation to support vqa by jaelgu in https://github.com/zilliztech/GPTCache/pull/264
* Update README by shiyu22 in https://github.com/zilliztech/GPTCache/pull/266
* Add evaluation options in vqa bootcamp by jaelgu in https://github.com/zilliztech/GPTCache/pull/267
* Add two streamlit demos. by wxywb in https://github.com/zilliztech/GPTCache/pull/268
* feat: Implement vit image embedding by Pouyanpi in https://github.com/zilliztech/GPTCache/pull/270
* Add `init_similar_cache` func for providing easier similar cache init by SimFG in https://github.com/zilliztech/GPTCache/pull/272
* Update sql example by shiyu22 in https://github.com/zilliztech/GPTCache/pull/273
* Update the version to `0.1.18` by SimFG in https://github.com/zilliztech/GPTCache/pull/275


**Full Changelog**: https://github.com/zilliztech/GPTCache/compare/0.1.17...0.1.18

🎉 Introduction to new functions of GPTCache

1. Add image embedding timm

python
import requests
from PIL import Image
from gptcache.embedding import Timm

url = 'https://raw.githubusercontent.com/zilliztech/GPTCache/main/docs/GPTCache.png'
image = Image.open(requests.get(url, stream=True).raw) Read image url as PIL.Image
encoder = Timm(model='resnet18')
image_tensor = encoder.preprocess(image)
embed = encoder.to_embeddings(image_tensor)


2. Add Replicate adapter, vqa (visual question answering) (**experimental**)

python
from gptcache.adapter import replicate

question = "what is in the image?"

replicate.run(
"andreasjansson/blip-2:xxx",
input={
"image": open(image_path, 'rb'),
"question": question
}
)


3. Support to flush data for preventing accidental loss of memory data

python
from gptcache import cache

cache.flush()


What's Changed
* Add image embedding timm by jaelgu in https://github.com/zilliztech/GPTCache/pull/249
* Fix typo in release_note.md by eltociear in https://github.com/zilliztech/GPTCache/pull/251
* Allow image path or file-like object as input of image embedding by jaelgu in https://github.com/zilliztech/GPTCache/pull/252
* Fix docs by junjiejiangjjj in https://github.com/zilliztech/GPTCache/pull/250
* Add replicate run adapter by shiyu22 in https://github.com/zilliztech/GPTCache/pull/248
* Fix adapter with image object store by jaelgu in https://github.com/zilliztech/GPTCache/pull/253
* Fix image mode issue by jaelgu in https://github.com/zilliztech/GPTCache/pull/254
* Fix that 'NoneType' object has no attribute 'answers' by SimFG in https://github.com/zilliztech/GPTCache/pull/256
* Update the bootcamp doc by shiyu22 in https://github.com/zilliztech/GPTCache/pull/257
* Support to flush data for preventing accidental loss of memory data by SimFG in https://github.com/zilliztech/GPTCache/pull/258
* Add evaluator for vqa by jaelgu in https://github.com/zilliztech/GPTCache/pull/259
* Enable adapter.replicate docstring in API reference by jaelgu in https://github.com/zilliztech/GPTCache/pull/260
* Update the version to `0.1.17` by SimFG in https://github.com/zilliztech/GPTCache/pull/261

New Contributors
* eltociear made their first contribution in https://github.com/zilliztech/GPTCache/pull/251

**Full Changelog**: https://github.com/zilliztech/GPTCache/compare/0.1.16...0.1.17

🎉 Introduction to new functions of GPTCache

1. Add StableDiffusion adapter (**experimental**)

python
import torch

from gptcache.adapter.diffusers import StableDiffusionPipeline
from gptcache.processor.pre import get_prompt
from gptcache import cache

cache.init(
pre_embedding_func=get_prompt,
)
model_id = "stabilityai/stable-diffusion-2-1"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)

prompt = "a photo of an astronaut riding a horse on mars"
pipe(prompt=prompt).images[0]


2. Add speech to text bootcamp, [link](https://github.com/zilliztech/GPTCache/tree/main/docs/bootcamp/openai/speech_to_text.ipynb)

3. More convenient management of cache files

python
from gptcache.manager.factory import manager_factory

data_manager = manager_factory('sqlite,faiss', data_dir="test_cache", vector_params={"dimension": 5})


4. Add a simple GPTCache server (**experimental**)

After starting this server, you can:

- put the data to cache, like: `curl -X PUT -d "receive a hello message" "http://localhost:8000?prompt=hello"`
- get the data from cache, like: `curl -X GET "http://localhost:8000?prompt=hello"`

Currently the service is just a map cache, more functions are still under development.

What's Changed
* Adapt StableDiffusion by jaelgu in https://github.com/zilliztech/GPTCache/pull/234
* Add audio embedding with data2vec by jaelgu in https://github.com/zilliztech/GPTCache/pull/238
* Support multi-model question by junjiejiangjjj in https://github.com/zilliztech/GPTCache/pull/235
* Add speech to text bootcamp by shiyu22 in https://github.com/zilliztech/GPTCache/pull/239
* Fix auto API references script for gptcache.adapter by jaelgu in https://github.com/zilliztech/GPTCache/pull/240
* Update README with multimodal adapter in modules by jaelgu in https://github.com/zilliztech/GPTCache/pull/242
* Add manager_factory to create data_manager by junjiejiangjjj in https://github.com/zilliztech/GPTCache/pull/241
* Add a simple GPTCache server by SimFG in https://github.com/zilliztech/GPTCache/pull/244


**Full Changelog**: https://github.com/zilliztech/GPTCache/compare/0.1.15...0.1.16