Marqo

New features
- Model authorisation(https://github.com/marqo-ai/marqo/pull/460). Non-public OpenCLIP and CLIP models can now be loaded
from Hugging Face and AWS s3 via the `model_location` settings object and `model_auth`.
See [here (model auth during search)](https://docs.marqo.ai/0.0.19/API-Reference/search/#model-auth)
and [here (model auth during add_documents)](https://docs.marqo.ai/0.0.19/API-Reference/documents/#model-auth) for usage.
- Max replicas configuration (https://github.com/marqo-ai/marqo/pull/465).
Marqo admins now have more control over the max number of replicas that can be set for indexes on the Marqo instance.
See [here](https://docs.marqo.ai/0.0.19/Advanced-Usage/configuration/#configuring-usage-limits) for how to configure this.

Breaking changes
- Marqo now allows for a maximum of 1 replica per index by default (https://github.com/marqo-ai/marqo/pull/465).

Bug fixes and minor changes
- README improvements (https://github.com/marqo-ai/marqo/pull/468)
- OpenCLIP version bumped (https://github.com/marqo-ai/marqo/pull/461)
- Added extra tests (https://github.com/marqo-ai/marqo/pull/464/)
- Unneeded files are now excluded in Docker builds (https://github.com/marqo-ai/marqo/pull/448, https://github.com/marqo-ai/marqo/pull/426)

Contributor shout-outs
- Thank you to our 2.9k stargazers!
- Thank you to community members for the increasingly exciting discussions on our Slack channel.
Feedback, questions and hearing about use cases helps us build a great open source product.
- Thank you to [jalajk24](https://github.com/jalajk24) for the PR to exclude unneeded files from Docker builds!

New features
- New E5 model type is available (https://github.com/marqo-ai/marqo/pull/419). E5 models are state of the art general-purpose text embedding models that obtained the best results on the MTEB benchmark when released in Dec 2022. Read more about these models [here](https://docs.marqo.ai/0.0.18/Models-Reference/dense_retrieval/#text).
- Automatic model ejection (https://github.com/marqo-ai/marqo/pull/372). Automatic model ejection helps prevent out-of-memory (OOM) errors on machines with a larger amount of CPU memory (16GB+) by ejecting the least recently used model.
- Speech processing article and example (https://github.com/marqo-ai/marqo/pull/431). [OwenPendrighElliott](https://github.com/OwenPendrighElliott) demonstrates how you can build and query a Marqo index from audio clips.

Optimisations
- Delete optimisation (https://github.com/marqo-ai/marqo/pull/436). The `/delete` endpoint can now handle a higher volume of requests.
- Inference calls can now execute in batches, with batch size configurable by an environment variable (https://github.com/marqo-ai/marqo/pull/376).

Bug fixes and minor changes
- Configurable max value validation for HNSW graph parameters (https://github.com/marqo-ai/marqo/pull/424). See [here](https://docs.marqo.ai/0.0.18/Advanced-Usage/configuration/#other-configurations) for how to configure.
- Configurable maximum number of tensor search attributes (https://github.com/marqo-ai/marqo/pull/430). See [here](https://docs.marqo.ai/0.0.18/Advanced-Usage/configuration/#other-configurations) for how to configure.
- Unification of vectorise output type (https://github.com/marqo-ai/marqo/pull/432)
- Improved test pipeline reliability (https://github.com/marqo-ai/marqo/pull/438, https://github.com/marqo-ai/marqo/pull/439)
- Additional image download tests (https://github.com/marqo-ai/marqo/pull/402, https://github.com/marqo-ai/marqo/pull/442)
- Minor fix in the Iron Manual example (https://github.com/marqo-ai/marqo/pull/440)
- Refactored HTTP requests wrapper (https://github.com/marqo-ai/marqo/pull/367)

Contributor shout-outs
- Thank you to our 2.8k stargazers!
- Thank you community members raising issues and discussions in our Slack channel.
- Thank you [jess-lord](https://github.com/jess-lord) and others for raising issues

New features
- New parameters that allow tweaking of Marqo indexes' underlying HNSW graph. `ef_construction` and `m` can be defined at index time (https://github.com/marqo-ai/marqo/pull/386, https://github.com/marqo-ai/marqo/pull/420, https://github.com/marqo-ai/marqo/pull/421), giving you more control over the relevancy/speed tradeoff. See usage and more details [here](https://docs.marqo.ai/0.0.17/API-Reference/indexes/#example_1).
- Score modification fields (https://github.com/marqo-ai/marqo/pull/414). Rank documents using knn similarity in addition to document metadata ( https://github.com/marqo-ai/marqo/pull/414). This allows integer or float fields from a document to bias a document's score during the knn search and allows additional ranking signals to be used. Use cases include giving more reputable documents higher weighting and de-duplicating search results. See usage [here](https://docs.marqo.ai/0.0.17/API-Reference/search/#score-modifiers).

Bug fixes and minor changes
- Added validation for unknown parameters during bulk search (https://github.com/marqo-ai/marqo/pull/413).
- Improved concurrency handling when adding documents to an index as it's being deleted (https://github.com/marqo-ai/marqo/pull/407).
- Better error messages for multimodal combination fields (https://github.com/marqo-ai/marqo/pull/395).
- Examples of recently added features added to README (https://github.com/marqo-ai/marqo/pull/403).

Contributor shout-outs
- Thank you to our 2.6k stargazers.
- Thank you to [anlrde](https://github.com/anlrde), [strich](https://github.com/strich), [feature-hope](https://github.com/feature-hope), [bazuker](https://github.com/bazuker) for raising issues!

New features
- Bulk search (https://github.com/marqo-ai/marqo/pull/363, https://github.com/marqo-ai/marqo/pull/373).
Conduct multiple searches with just one request. This improves search throughput in Marqo by parallelising multiple search queries in a single API call.
The average search time can be decreased up to 30%, depending on your devices and models.
Check out the usage guide [here](https://docs.marqo.ai/0.0.16/API-Reference/bulk)
- Configurable number of index replicas (https://github.com/marqo-ai/marqo/pull/391).
You can now configure how many replicas to make for an index in Marqo using the `number_of_replicas` parameter. Marqo makes 1 replica by default.
We recommend having at least one replica to prevent data loss.
See the usage guide [here](https://docs.marqo.ai/0.0.16/API-Reference/indexes/#body-parameters)
- Use your own vectors during searches (https://github.com/marqo-ai/marqo/pull/381). Use your own vectors as context for your queries.
Your vectors will be incorporated into the query using a weighted sum approach,
allowing you to reduce the number of inference requests for duplicated content.
Check out the usage guide [here](https://docs.marqo.ai/0.0.16/API-Reference/search/#context)

Bug fixes and minor changes
- Fixed a bug where some Open CLIP models were unable to load checkpoints from the cache (https://github.com/marqo-ai/marqo/pull/387).
- Fixed a bug where multimodal search vectors are not combined based on expected weights (https://github.com/marqo-ai/marqo/pull/384).
- Fixed a bug where multimodal document vectors are not combined in an expected way. `numpy.sum` was used rather than `numpy.mean`. (https://github.com/marqo-ai/marqo/pull/384).
- Fixed a bug where an unexpected error is thrown when `using_existing_tensor = True` and documents are added with duplicate IDs (https://github.com/marqo-ai/marqo/pull/390).
- Fixed a bug where the index settings validation did not catch the `model` field if it is in the incorrect part of the settings json (https://github.com/marqo-ai/marqo/pull/365).
- Added missing descriptions and requirement files on our [GPT-examples](https://github.com/marqo-ai/marqo/tree/mainline/examples/GPT-examples) (https://github.com/marqo-ai/marqo/pull/349).
- Updated the instructions to start Marqo-os (https://github.com/marqo-ai/marqo/pull/371).
- Improved the Marqo start-up time by incorporating the downloading of the punkt tokenizer into the dockerfile (https://github.com/marqo-ai/marqo/pull/346).

Contributor shout-outs
- Thank you to our 2.5k stargazers.
- Thank you to [ed-muthiah](https://github.com/ed-muthiah) for submitting a PR (https://github.com/marqo-ai/marqo/pull/349)
that added missing descriptions and requirement files on our [GPT-examples](https://github.com/marqo-ai/marqo/tree/mainline/examples/GPT-examples).

New features
- Multimodal tensor combination (https://github.com/marqo-ai/marqo/pull/332, https://github.com/marqo-ai/marqo/pull/355). Combine image and text data into a single vector! Multimodal combination objects can be added as Marqo document fields. For example, this can be used to encode text metadata into image vectors. See usage [here](https://docs.marqo.ai/0.0.15/Advanced-Usage/document_fields/#multimodal-combination-object).

Bug fixes
- Fixed a bug that prevented CLIP's device check from behaving as expected (https://github.com/marqo-ai/marqo/pull/337)
- CLIP utils is set to use the OpenCLIP default tokenizer so that long text inputs are truncated correctly (https://github.com/marqo-ai/marqo/pull/351).

Contributor shout-outs:
- Thank you to our 2.4k stargazers
- Thank you to [ed-muthiah](https://github.com/ed-muthiah), [codebrain](https://github.com/codebrain) and others for raising issues.

New features
- `use_existing_tensors` flag, for `add_documents` (https://github.com/marqo-ai/marqo/pull/335). Use existing Marqo tensors to autofill unchanged tensor fields, for existing documents. This lets you quickly add new metadata while minimising inference operations. See usage [here](https://docs.marqo.ai/0.0.14/API-Reference/documents/#query-parameters).
- `image_download_headers` parameter for `search` and `add_documents` (https://github.com/marqo-ai/marqo/pull/336). Index and search non-publicly available images. Add image download auth information to `add_documents` and `search` requests. See usage [here](https://docs.marqo.ai/0.0.14/API-Reference/image_downloads/).

Optimisations
- The index cache is now updated on intervals of 2 seconds (https://github.com/marqo-ai/marqo/pull/333), rather than on every search. This reduces the pressure on Marqo-OS, allowing for greater search and indexing throughput.

Bug fixes
- Helpful validation errors for invalid index settings (https://github.com/marqo-ai/marqo/pull/330). Helpful error messages allow for a smoother getting-started experience.
- Automatic precision conversion to `fp32` when using `fp16` models on CPU (https://github.com/marqo-ai/marqo/pull/331).
- Broadening of the types of image download errors gracefully handled. (https://github.com/marqo-ai/marqo/pull/321)