Tigerbeetle

Released: 2024-11-18

Safety And Performance

- [2461](https://github.com/tigerbeetle/tigerbeetle/pull/2461)

Fix a broken assert when a recently-state-synced replica that has not completed journal repair
receives an old `commit` message.

- [2474](https://github.com/tigerbeetle/tigerbeetle/pull/2474)

Retry `EAGAIN` on (disk) reads. This is essential for running TigerBeetle on XFS, since XFS
returns `EAGAIN` unexpectedly.

- [2476](https://github.com/tigerbeetle/tigerbeetle/pull/2476)

Fix a message bus crash when a client reconnects to a replica without the replica receiving a
disconnect for the first connection.

- [2475](https://github.com/tigerbeetle/tigerbeetle/pull/2475)

Save 256KiB of RAM by not having a prefetch cache for historical balances.
(Historical balances are never prefetched, so this cache was unused.)

- [2482](https://github.com/tigerbeetle/tigerbeetle/pull/2482)

Update hardware requirements in the documentation to include the recommended network bandwidth,
advice for very large data files, and farther emphasis on the importance of ECC RAM.

- [2484](https://github.com/tigerbeetle/tigerbeetle/pull/2484)

Don't panic the client when the client's session is
[evicted](https://docs.tigerbeetle.com/reference/sessions/#eviction).
Instead, report an error any time a new batch is submitted to the evicted client.
(How the error is reported depends on the client language – e.g. Java throws an exception, whereas
Node.js rejects the `Promise`).

Note that if running clients are evicted, that typically indicates that there are too many clients
running – check out the
[suggested system architecture](https://docs.tigerbeetle.com/coding/system-architecture/).

Features

- [2464](https://github.com/tigerbeetle/tigerbeetle/pull/2464)

Add REPL interactivity. Also change the REPL from dynamic to static allocation.
Thanks wpaulino!

Internals

- [2481](https://github.com/tigerbeetle/tigerbeetle/pull/2481)

Expose the VSR timestamp to the client. (This is an experimental feature which will be removed
soon – don't use this!)

TigerTracks 🎧

- [Olympic Airways](https://www.youtube.com/watch?v=4BcNLA2KB2c)

Released: 2024-11-11

Safety And Performance

- [2435](https://github.com/tigerbeetle/tigerbeetle/pull/2435)

Fix an attempt to access uninitialized fields of `tb_packet_t` when `tb_client_deinit` aborts
pending requests. Also add Java unit tests to reproduce the problem and validate the fix.

- [2437](https://github.com/tigerbeetle/tigerbeetle/pull/2437)

Fix a liveness issue where the cluster gets stuck despite sufficient durability, caused by buggy
logic for cycling through faulty blocks during repair. Now, we divide the request buffer between
the `read_global_queue` and `faulty_blocks`, ensuring that we always request blocks from both.

Features

- [2462](https://github.com/tigerbeetle/tigerbeetle/pull/2462)

Update DevHub styling.

- [2424](https://github.com/tigerbeetle/tigerbeetle/pull/2424)

Vortex can now not only crash replicas (by killing and restarting the process) but also stop and
resume them.

Internals

- [2458](https://github.com/tigerbeetle/tigerbeetle/pull/2458)

Fix the Dotnet walkthrough example that misused `length` instead of the final index when slicing
an array. Thanks tenatus for reporting it!

- [2453](https://github.com/tigerbeetle/tigerbeetle/pull/2453)

Fix a CI failure caused by concurrent processes trying to create the `fs_supports_direct_io`
probe file in the same path.

- [2441](https://github.com/tigerbeetle/tigerbeetle/pull/2441)

Replace curl shell invocation with Zig's http client. 😎

- [2454](https://github.com/tigerbeetle/tigerbeetle/pull/2454)

Properly handle "host unreachable" (`EHOSTUNREACH`) on Linux, instead of returning unexpected
error.

- [2443](https://github.com/tigerbeetle/tigerbeetle/pull/2443),
[2451](https://github.com/tigerbeetle/tigerbeetle/pull/2451),
[2459](https://github.com/tigerbeetle/tigerbeetle/pull/2459),
[2460](https://github.com/tigerbeetle/tigerbeetle/pull/2460),
[2465](https://github.com/tigerbeetle/tigerbeetle/pull/2465)

Various code refactorings to improve naming conventions, readability, and organization.

TigerTracks 🎧

- [Scatman (Ski-ba-bop-ba-dop-bop)](https://www.youtube.com/watch?v=ZhSY7vYbXkk&t=106s)

TigerBeetle (unreleased)

Released: 2024-11-04

Safety And Performance

- [2356](https://github.com/tigerbeetle/tigerbeetle/pull/2356)

Add "Vortex" – a full-system integration test.
Notably, unlike the VOPR this test suite covers the language clients.

- [2430](https://github.com/tigerbeetle/tigerbeetle/pull/2430)

Cancel in-flight async (Linux) IO before freeing memory. This was not an issue on the replica
side, as replicas only stop when their process stops. However, clients may be closed
without the process also ending. If IO is still in flight when this occurs, we must ensure that
all IO is cancelled before the client's buffers are freed, to guard against a use-after-free.

This PR also fixes an unrelated assertion failure that triggered when closing a client that
had already closed its socket.

- [2432](https://github.com/tigerbeetle/tigerbeetle/pull/2432)

On startup and after checkpoint, assert that number of blocks acquired by the free set is
consistent with the number of blocks we see acquired via the manifest and checkpoint trailers.

- [2436](https://github.com/tigerbeetle/tigerbeetle/pull/2436)

Reject connections from unknown replicas.

- [2438](https://github.com/tigerbeetle/tigerbeetle/pull/2438)

Fix multiversion builds on MacOS.

- [2442](https://github.com/tigerbeetle/tigerbeetle/pull/2442)

On an unrecognized error code from the OS, print that error before we panic.
(This was already the policy in `Debug` builds, but now it includes `ReleaseSafe` as well.)

- [2444](https://github.com/tigerbeetle/tigerbeetle/pull/2444)

Fix a panic involving an in-flight write to an old reply after state sync.

Internals

- [2440](https://github.com/tigerbeetle/tigerbeetle/pull/2440)

Expose reply timestamp from `vsr.Client`.
(Note that this is not yet surfaced by language clients).

TigerTracks 🎧

- [Everything In Its Right Place](https://www.youtube.com/watch?v=onRk0sjSgFU)

Released: 2024-10-28

Safety And Performance

- [2428](https://github.com/tigerbeetle/tigerbeetle/pull/2428)

Make `Grid.reserve()` abort rather than returning null.
When `Grid.reserve()` aborts, that indicates that the data file size limit would be exceeded by
the reservation. We were already panicking in this case by unwrapping the result, but now it has
a useful error message.

- [2416](https://github.com/tigerbeetle/tigerbeetle/pull/2416)

Improve availability and performance by sending `start_view` message earlier in the new-primary
recovery – as soon as the journal headers are repaired.

- [2360](https://github.com/tigerbeetle/tigerbeetle/pull/2360)

Refactor compaction to clarify the scheduling logic, schedule more aggressively, and make it
easier to run multiple compactions concurrently. This also improved the benchmark performance.

Features

- [2425](https://github.com/tigerbeetle/tigerbeetle/pull/2425)

Support multiversion (non-automatic) upgrades when the replica is started with `--development`
or `--experimental`.

Internals

- [2427](https://github.com/tigerbeetle/tigerbeetle/pull/2427)

Allow a release's Git tag and `config.process.release` to differ. This simplifies the release
process for hotfixes, when the Git tag is bumped but the `config.process.release` is unchanged.

TigerTracks 🎧

- [Stuck in a Timeloop](https://www.youtube.com/watch?v=FWBjzQnDb8o)

Released: 2024-10-21

Safety And Performance

- [2414](https://github.com/tigerbeetle/tigerbeetle/pull/2414)

Improve performance & availability during view change by ensuring a replica only repairs the
portion of the WAL that is *required* to become primary, instead of repairing it in its entirety.

- [2412](https://github.com/tigerbeetle/tigerbeetle/pull/2412)

Add a unit test for Zig's stdlib sort.

Stable sort is critical for compaction correctness. Zig stdlib does have a sort fuzz test, but it
doesn't cover the presorted subarray case, and doesn't check arrays much larger than the sort
algorithm's on-stack cache.

- [2413](https://github.com/tigerbeetle/tigerbeetle/pull/2413)

Fix a bug in the MessageBus wherein connections weren't being terminated during client teardown.

Internals


- [2405](https://github.com/tigerbeetle/tigerbeetle/pull/2405)

Fix a bug in the benchmark wherein the usage of `--account-count-hot` was broken when used in
conjunction with the `uniform` distribution.

- [2409](https://github.com/tigerbeetle/tigerbeetle/pull/2409)

Revamp the `core_missing_prepares` liveness-mode check to correctly check for the prepares that a
replica should repair (after [2414](https://github.com/tigerbeetle/tigerbeetle/pull/2414)).


TigerTracks 🎧

- [Last Train Home](https://open.spotify.com/track/0tgBtQ0ISnMQOKorrN9HLX?si=8a05ab879730455b)

Released: 2024-10-15

Safety And Performance

- [2394](https://github.com/tigerbeetle/tigerbeetle/pull/2394),
[2401](https://github.com/tigerbeetle/tigerbeetle/pull/2401)

TigerBeetle clients internally batch operations for improved performance. Fix a bug where an
unclosed link chain could be batched before another linked chain, causing them to be treated as
one long linked chain. Additionally, prevent non-batchable requests from sharing packets entirely.

- [2398](https://github.com/tigerbeetle/tigerbeetle/pull/2398)

`AMOUNT_MAX` is used as a sentinel value for things like balancing transfers to specify moving
as much as possible. Correct and fix its value in the Java client. Thanks tKe!

Features

- [2274](https://github.com/tigerbeetle/tigerbeetle/pull/2274)

Improve the benchmark by adding Zipfian distributed random numbers, to better simulate realistic
conditions and as a precursor to approximating YCSB.

- [2393](https://github.com/tigerbeetle/tigerbeetle/pull/2393)

Previously, TigerBeetle's clients disallowed empty batches locally, before the request was even
sent to the cluster. However, this is actually a valid protocol message - even if it's not used by
the current state machine - so allow empty batches to be sent from clients.

- [2384](https://github.com/tigerbeetle/tigerbeetle/pull/2384)

Revamp client documentation so that each snippet is self-contained, and standardize it across all
languages.

Internals

- [2404](https://github.com/tigerbeetle/tigerbeetle/pull/2404)

Give the [DevHub](https://tigerbeetle.github.io/tigerbeetle/) a fresh coat of paint, and fix
passing seeds being blue in dark mode.

- [2408](https://github.com/tigerbeetle/tigerbeetle/pull/2408),
[2400](https://github.com/tigerbeetle/tigerbeetle/pull/2400),
[2391](https://github.com/tigerbeetle/tigerbeetle/pull/2391),
[2399](https://github.com/tigerbeetle/tigerbeetle/pull/2399),
[2402](https://github.com/tigerbeetle/tigerbeetle/pull/2402),
[2385](https://github.com/tigerbeetle/tigerbeetle/pull/2385)

Improve VOPR logging and fix a few failing seeds.

TigerTracks 🎧

- [99 Luftballons](https://www.youtube.com/watch?v=Fpu5a0Bl8eY)

Released: 2024-10-07

Safety And Performance

- [2359](https://github.com/tigerbeetle/tigerbeetle/pull/2359)

Significantly reduced P100 latency by incrementally spreading the mutable table's sort during
compaction. This leverages the optimization of sort algorithms for processing sequences of
already sorted sub-arrays.

- [2367](https://github.com/tigerbeetle/tigerbeetle/pull/2367)

Improve the workload generator to support concurrent tests with different ledgers.

- [2382](https://github.com/tigerbeetle/tigerbeetle/pull/2382),
[2363](https://github.com/tigerbeetle/tigerbeetle/pull/2363)

Fix VOPR seeds.
For more awesome details about the backstory and solutions to these issues,
please refer to the PR.

Features

- [2358](https://github.com/tigerbeetle/tigerbeetle/pull/2358)

Update the REPL to support representing the maximum integer value as `-0`,
serving as the `AMOUNT_MAX` sentinel.
Additionally, other negative values such as `-1` can be used to represent `maxInt - 1`.

Also, include support for hexadecimal numbers for more convenient inputting of GUID/UUID
literals (e.g. `0xa1a2a3a4_b1b2_c1c2_d1d2_e1e2e3e4e5e6`).

Allow the `timestamp` field to be set, enabling the REPL to be used for `imported` events.

Internals

- [2376](https://github.com/tigerbeetle/tigerbeetle/pull/2376)

Use `zig fetch` as a replacement for downloading files, removing dependence on external tools.

- [2383](https://github.com/tigerbeetle/tigerbeetle/pull/2383)

Port of Rust's [`dbg!`](https://doc.rust-lang.org/std/macro.dbg.html) macro to Zig,
and the corresponding CI validation to prevent code using it from being merged into `main`! 😎

- [2370](https://github.com/tigerbeetle/tigerbeetle/pull/2370),
[2373](https://github.com/tigerbeetle/tigerbeetle/pull/2373)

Verify the release versions included in the multiversion binary pack at build time (not only
during runtime) and improve the `tigerbeetle version --verbose` command's `multiversion` output.

- [2369](https://github.com/tigerbeetle/tigerbeetle/pull/2369)

Fix a multiversioning issue where the binary size exceeded the read buffer, failing to parse the
executable header.

- [2380](https://github.com/tigerbeetle/tigerbeetle/pull/2380)

Consistently use `transient_error` instead of `transient_failure` and cleanup the StateMachine
code.

- [2379](https://github.com/tigerbeetle/tigerbeetle/pull/2379)

Add missing links to the operations `query_accounts` and `query_transfers` in the documentation
and include the declaration for `QueryFilter` and `QueryFilterFlags` in the `tb_client.h` header.

- [2333](https://github.com/tigerbeetle/tigerbeetle/pull/2333)

Clearer error message when the replica crashes due to a data file being too large, instructing
the operator to increase the memory allocated for the manifest log.

TigerTracks 🎧

- [Creep](https://www.youtube.com/watch?v=XFkzRNyygfk)