Typesense

This release contains new features, performance improvements and important bug fixes.

New Features

- **Indexing of nested object fields**: feature must be enabled via the `enable_nested_fields`
option during collection creation.
- **Cross-field OR and complex filter expressions**: a filter query like
`is_premium: true || (price:> 100 && category: shoes)` is now possible.
- **Vector search**: support for both exact & HNSW-based approximate vector searching.
- **Multi-lingual support**: Chinese, Japanese, Korean, Cyrillic and Thai are now supported via the use of field-level
`locale` flag in the collection schema.
- **Optional filtering**: `sort_by` clause can now accept an expression whose result is used for sorting, e.g.
`sort_by=_eval(brand:nike):desc,_text_match:desc`.
- **Preset search configurations**: manage your search query parameters by storing them within Typesense as a "preset",
that you can refer to during query time. This helps in keeping the query parameters hidden away from public view.

Enhancements

- New `text_match_type` parameter that allows you to customize how multi-field text relevancy score is computed.
- Improve performance of large collection deletions.
- Ability to clone a collection schema (without documents), overrides and synonyms.
- New highlight structure that mimics the original document structure. Nested fields are highlighted only in this new
structure, which is returned in a key named `highlight` in the JSON response.
- Allow override rules to be processed past the first match via the `stop_processing` flag (default is `true`).
- Support locale and symbols in synonyms via the `locale` and `symbols_to_index` options during synonym creation.
- Allow cloning of collection schema & linked assets (like synonyms, overrides) from a reference collection.
- Support query replacement action in an override via the `replace_query` option.
- Support an override to be active with a given time window via the `effective_from_ts` and `effective_to_ts` options.
- Support `filter_by` rule in overrides.
- New `--skip-writes` flag for starting Typesense in a mode that does not read writes from the Raft log. This is
useful when the server has crashed due to some recent bad writes that you want to skip over temporarily.
- New `--memory-used-max-percentage` and `--disk-used-max-percentage` flags for preventing writes when a specified
memory/disk threshold is breached.
- Allow the imported documents and their `id`s to be returned in the import response via
the `return_doc` and `return_id` options.
- API for compacting the on-disk database via the `POST /operations/db/compact` end-point.

Bug Fixes

- Fixed some edge cases with schema alteration.
- Fixed a race condition in parallel collection creation that manifested on localhost, especially on Mac.
- Fixed an edge case in highlighting involving non-ASCII unicode characters.
- Fixed sort by string not accounting for accented characters.
- Fixed an edge case in numerical facet field values not being fully removed on deletion.
- Fixed HTTPS POST body buffering removing genuine trailing white space in the chunked payload in some cases.
- Fixed float field validation to handle scientific notation.
- Fixed a bug involving exact filter match on array.
- Fixed a few edge cases that showed up in super-large documents that contained greater than 64,000 tokens.
- Implemented automatic log rotation for RocksDB info logs. Previously, one had to restart the server for truncation or
perform external truncation via `logrotate`.

Deprecations / behavior changes

- The default value for `search_cutoff_ms` is now `30000`, i.e. 30 seconds. If you wish to allow searches to run
longer than that, you can pass a higher value as a search parameter.
- Disable text match score bucketing, if there are more buckets than the number of results found. Previously, results
were being aggregated into a single score.
- The older `highlights` key in the response is deprecated (but is still returned for now). Use the new `highlight`
object in the response for highlighting information.
- The `text_match` key in the response is deprecated (but is still returned for now) in favor of the
new `text_match_info` object that returns fine grain matching information, including the score.

Upgrade

Before upgrading your existing Typesense cluster to `v0.24.0`, please review the behavior changes above to prepare your application for the upgrade.

- For self-hosted Typesense deployment, please refer to the important upgrade section of the [0.24.0 documentation](https://typesense.org/docs/0.24.0/api/#upgrading). This particular version requires a specific set of upgrade steps especially for multi-node clusters.
- For Typesense Cloud, visit your [clusters page](https://cloud.typesense.org/clusters), click on the cluster you want to upgrade, click on "Modify Configuration" on the right pane and schedule a time for your upgrade. Alternatively, you can also email `supporttypesense.org`.

This release contains a few enhancements and bug fixes identified in [v0.23.0](https://github.com/typesense/typesense/releases/tag/v0.23.0) which we thought are important enough to warrant a patch release. If you're using v0.23.0, we recommend upgrading.

Bug Fixes

- Fixed a potential crash caused by attempting to alter the `id` field.
- Fixed synonym queries not respecting the `prefix` setting.
- Fixed a rare bug present in exact filtering of string arrays.
- Fixed a race condition in slow request logging that led to a crash under high concurrency.
- Better validation of regular expression used in API key collection allow list.
- ARM builds now work on more ARM processors (previously worked only on Graviton instances)

Enhancements

- Improved filtering performance, upto 5x-10x faster when querying a small subset on large datasets
- Support for `sort_by` parameter in override rule.
- Allow word position in a field value to be used a ranking signal via the `prioritize_token_position` search parameter.
- Improved stability in rotation of geographically distributed clusters having a large dataset.

Download

Please download the appropriate binary archive for your operating system and architecture here: https://typesense.org/downloads/

**NOTE:** The new version is also available on [Typesense Cloud](https://cloud.typesense.org/). If you're already using Typesense Cloud, please reach out to us to have your cluster upgraded to v0.23.1.

Upgrade

Before upgrading your existing Typesense cluster to v0.23.1, please review the changes above to prepare your application for the upgrade.

- ⚠️ For self-hosted Typesense deployment, please refer to the important upgrade section of the [0.23 documentation](https://typesense.org/docs/0.23.1/api/#upgrading).
- For Typesense Cloud, email us at `contacttypesense.org` and we can do an in-place upgrade for you.

This release contains new features, performance improvements and important bug fixes.

New Features

- Phrase search: wrap keywords in a query with double quotes to search them as a phrase, e.g. `"new york"`.
- Schema changes: we now support fields to be added or dropped from a collection schema in-place.
- Improved multi-field matching: much better performance and accuracy when query keywords have to be matched across
multiple fields in a document.
- Infix searching: find string within strings, which is useful for entities like model number or email address.
- Allow string fields to be sorted: sorting on string fields can be enabled by setting `sort: true` parameter of the field.

Enhancements

- Improved update and delete performance of numerical fields by 10x.
- Emplace mode for imports: using the `emplace` action creates a document if it does not exist in a collection
or updates it (partially or fully) if it already exists.
- Treat space as typo: search for `basket ball` if `basketball` is not found or vice-versa. You can disable this behavior
by setting `split_join_tokens: false`.
- Improved Cyrillic support: better highlighting and fuzzy search for fields configured with:
`el`, `ru`, `sr`, `uk` and `be` locales.
- ARM compatibility: an ARM build is now published for every release.
- Each multi-search query can have an independent `x-typesense-api-key` key. This is useful to specify different scoped search API keys for each search.
- Control the number of words that Typesense considers for typo and prefix searching via the `max_candidates` parameter.
- CORS can now be enabled for a specific set of domains using the `--cors-domains` flag.
- Search results are now highlighted by prefix, rather than the full world.
Previously, searching for "app" will highlight the full word "apple" in the results, but now it will only highlight the "app"le prefix within the word.
- "Remove Matched Tokens" can be used by itself in Overrides. So you can now setup rules like, if query contains "word", remove "word" from the search query.
- Ability to toggle if filters should by applied to overrides or not using the `filter_curated_hits` flag.
- Ability to hide `out_of` and `search_time_ms` from the search API response, using the `exclude_fields` parameter.
- Ability to control typo tolerance for facet queries using `facet_query_num_typos`.
- Ability to specify which subnet to use for peering using `--peering-subnet` server parameter.

Bug Fixes

- Fixed exact match of synonym query candidates not being ranked correctly.
- Fix glibc incompatibility on recent Linux distros (Ubuntu 21.04+). [531](https://github.com/typesense/typesense/issues/531)
- Fixed the `snippet` containing the full field value when `highlight_full_fields` is enabled.
- Fixed `--enable-cors=true` flag format not working. Earlier, only the `--enable-cors` format worked.
- Fixed exact match for repeated words (when searching for repeated words such as "Boom Boom"). [427](https://github.com/typesense/typesense/issues/427)
- Fixed `highlight_fields` parameter not respecting `include_fields`. [556](https://github.com/typesense/typesense/issues/556)
- Fixed document ids that are accepted with space char (%20) but cannot be deleted later. [574](https://github.com/typesense/typesense/issues/574)
- Improved highlighting of text containing punctuations. [528](https://github.com/typesense/typesense/issues/528)
- Fixed case sensitivity of facet fields. [504](https://github.com/typesense/typesense/issues/504)

Deprecations / behavior changes

- In prefix queries, only the prefix part of a word in the result is highlighted now, instead of the whole word.
For e.g. given a query like "new y", the result will be highlighted as `<mark>New Y</mark>ork City`.

Download

Please download the appropriate binary archive for your operating system and architecture here: https://typesense.org/downloads/

**NOTE:** The new version is also available on [Typesense Cloud](https://cloud.typesense.org/). If you're already using Typesense Cloud, please reach out to us to have your cluster upgraded to v0.23.0.

Upgrade

Before upgrading your existing Typesense cluster to v0.23.0, please review the behavior changes above to prepare your application for the upgrade.

- For self-hosted Typesense deployment, please refer to the important upgrade section of the [0.23.0 documentation](https://typesense.org/docs/0.23.0/api/#upgrading). This particular version requires a specific set of upgrade steps especially for multi-node clusters.
- For Typesense Cloud, visit your [clusters page](https://cloud.typesense.org/clusters), click on the cluster you want to upgrade, click on "Modify Configuration" on the right pane and schedule a time for your upgrade. Alternatively, you can also email `supporttypesense.org`.

This release contains a couple of bug fixes identified in [`v0.22.1`](https://github.com/typesense/typesense/releases/tag/v0.22.1) which we thought are important enough to warrant a patch release. If you're using `v0.22.1`, we recommend upgrading.

Bug Fixes

- Handle bad geo polygon vertices (e.g. duplicate points).
- Fixed an edge case in ranking of words that share a prefix during prefix search.
- Better validation + handling of unexpected data errors during indexing.
- Fixed a rare but critical bug that manifested during document updates that had performance implications.

Download

Please download the appropriate binary archive for your operating system and architecture here: https://typesense.org/downloads/

**NOTE:** The new version is also available on [Typesense Cloud](https://cloud.typesense.org/). If you're already using Typesense Cloud, please reach out to us to have your cluster upgraded to v0.22.2.

Upgrade

Before upgrading your existing Typesense cluster to v0.22, please review the behavior changes above to prepare your application for the upgrade.

- ⚠️ For self-hosted Typesense deployment, please refer to the important upgrade section of the [0.22 documentation](https://typesense.org/docs/0.22.2/api/#upgrading).
- For Typesense Cloud, email us at `contacttypesense.org` and we can do an in-place upgrade for you.

This release contains a couple of bug fixes identified in [`v0.22.0`](https://github.com/typesense/typesense/releases/tag/v0.22.0) which we thought are important enough to warrant a patch release. If you're using `v0.22.0`, we recommend upgrading.

Bug Fixes

- Fixed an edge case in exporting of documents using a `filter_by` condition: documents were being duplicated.
- Allow a document to contain a dict/hashmap field when a wildcard auto (`.*`) field is present in the collection schema.

Download

Please download the appropriate binary archive for your operating system and architecture here: https://typesense.org/downloads/

**NOTE:** The new version is also available on [Typesense Cloud](https://cloud.typesense.org/). If you're already using Typesense Cloud, please reach out to us to have your cluster upgraded to v0.22.1.

Upgrade

Before upgrading your existing Typesense cluster to v0.22, please review the behavior changes above to prepare your application for the upgrade.

- ⚠️ For self-hosted Typesense deployment, please refer to the important upgrade section of the [0.22 documentation](https://typesense.org/docs/0.22.1/api/#upgrading).
- For Typesense Cloud, email us at `contacttypesense.org` and we can do an in-place upgrade for you.

This release contains new features, performance improvements and important bug fixes.

New Features

- Customizable word separators: define a list of special characters via the `token_separators` configuration
during schema creation. These characters are then used as word separators, _in addition_ to space and new-line characters.
- Index and search special characters: define a list of special characters that will be indexed as text via the
`symbols_to_index` configuration during schema creation.
- Dynamic filtering based on rules: overrides now support a `filter_by` clause that can apply filters dynamically to query rules defined in the override.
- Server side caching: cache search requests for a configurable amount of time to improve perceived latencies on heavy queries. Refer to the `use_cache` and `cache_ttl` parameters. By default, caching is disabled.
- Protection against expensive queries via the use of `search_cutoff_ms` parameter that attempts to return results early when the cutoff time has elapsed. This is not a strict guarantee and facet computation is not bound by this parameter.
- Added `geo_precision` sorting option to geo fields. This will bucket geo points into "groups" determined by the given precision value, such that points that fall within the same group are treated as equal, and the next sorting field can be considered for ranking.

Enhancements

- Reduced memory consumption: 20-30% depending on the shape of your data.
- Improved update performance: updates on string fields are now 5-6x faster.
- Improved search performance: 20-25% faster on a variety of datasets we tested on.
- Improved parallelization for multi-collection writes: collections are now indexed independently, making indexing much faster when you are writing to hundreds of collections at the same time.
- Allow exhaustive searching via the `exhaustive_search` parameter. Setting `?exhaustive_search=true` will make Typesense consider _all_ prefixes and typo corrections of the words in the query without stopping early when enough results are found (`drop_tokens_threshold` and `typo_tokens_threshold` configurations are ignored).
- Exact filtering on strings (using the `:=` operator) no longer requires the field to be defined as a facet.
- Make minimum word length for 1-char typo and 2-char typos configurable via `min_len_1typo` and `min_len_2typo` parameters. Defaults are 4 and 7 respectively.
- Support filtering by document `id` in filter_by query.
- Support API key permission for creating a specific collection: previously, there was no way to generate an API key that allows you to create a collection with a specific name.
- Allow use of type `auto` for a field whose name does not contain a regular expression.
- Geosearch polygon filter automatically sorts the geo points for the polygon in the correct order: so you don't have to define them in counter clockwise order anymore.

Bug Fixes

- Fixed edge cases in import of large documents where sometimes, imports hanged mysteriously or ended prematurely.
- Fixed document with duplicate IDs within an import upsert batch being imported as two separate documents.
- Fixed fields with names that contain a regular expression acting as an `auto` type instead of respecting the schema type.
- Fixed a few edge cases in multi-field searching, especially around field weighting and boosting.
- Fixed deletion of collections with slashes or spaces in their names not working: you can now URL encode the names while calling the API.

Deprecations / behavior changes

- Once you upgrade your Typesense server to `v0.22`, the data directory cannot be used with
`v0.21.0` binary again. So, please take a snapshot/backup of the data directory before upgrading. See important upgrade instructions below.
- The `drop_tokens_threshold` and `typo_tokens_threshold` now default to a value of `1`.
If you were relying on the earlier defaults (`10` and `100` respectively), please set these parameters explicitly.
- Minimum word length for 1-char typo correction has been increased to 4.
Likewise, minimum length for 2-char typo has been increased to 7. This has helped to reduce false fuzzy matches.
You can use the `min_len_1typo` and `min_len_2typo` parameters to customize these default values.

Download

Please download the appropriate binary archive for your operating system and architecture here: https://typesense.org/downloads/

**NOTE:** The new version is also available on [Typesense Cloud](https://cloud.typesense.org/). If you're already using Typesense Cloud, please reach out to us to have your cluster upgraded to v0.22.0.

Upgrade

Before upgrading your existing Typesense cluster to v0.22, please review the behavior changes above to prepare your application for the upgrade.

- For self-hosted Typesense deployment, please refer to the important upgrade section of the [0.22 documentation](https://typesense.org/docs/0.22.0/api/#upgrading). This particular version requires a specific set of upgrade steps especially for multi-node clusters.
- For Typesense Cloud, email us at `contacttypesense.org` and we can do an in-place upgrade for you.